Abstract

Multi-agent path planning (MAPP) is crucial for large-scale mobile robot systems to work safely and properly in complex environments. Existing learning-based decentralized MAPP approaches allow each agent to gather information from nearby agents, leading to more efficient coordination among agents. However, these approaches often struggle with reasonably handling local information inputs for each agent, and their communication mechanisms between agents need to be further refined to treat those congested traffic scenarios effectively. To address these issues, we propose a decentralized MAPP approach based on imitation learning and selective communication. Our approach adopts an imitation learning architecture that enables agents to rapidly learn complex behaviors from expert planning experience. The information extraction layer is integrated with convolutional neural network (CNN) and gated recurrent unit (GRU) for capturing features from local field-of-view observations. A two-stage selective communication process based on graph attention neural network (GAT) is developed to reduce the required neighbor agents in inter-agent communication. In addition, an adaptive strategy switching mechanism utilizing local expert-planned paths is designed to support robots to escape from local traps. The effectiveness of our proposed approach is evaluated in simulated grid environments with varying map sizes, obstacle densities, and numbers of agents. Experimental results show that our approach outperforms other decentralized path planning methods in success rate while maintaining the lowest flowtime variation and communication frequency. Furthermore, our approach is computationally efficient and scalable, making it suitable for real-world applications.

Graphical Abstract Figure
Graphical Abstract Figure
Close modal

References

1.
De Ryck
,
M.
,
Versteyhe
,
M.
, and
Debrouwere
,
F.
,
2020
, “
Automated Guided Vehicle Systems, State-of-the-Art Control Algorithms and Techniques
,”
J. Manuf. Syst.
,
54
, pp.
152
173
.
2.
Poudel
,
L.
,
Blair
,
C.
,
McPherson
,
J.
,
Sha
,
Z.
, and
Zhou
,
W.
,
2020
, “
A Heuristic Scaling Strategy for Multi-Robot Cooperative Three-Dimensional Printing
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
4
), p. 041002.
3.
Ghassemi
,
P.
, and
Chowdhury
,
S.
,
2020
, “
An Extended Bayesian Optimization Approach to Decentralized Swarm Robotic Search
,”
ASME J. Comput. Inf. Sci. Eng.
,
20
(
5
), p.
051003
.
4.
Sharon
,
G.
,
Stern
,
R.
,
Felner
,
A.
, and
Sturtevant
,
N. R.
,
2015
, “
Conflict-Based Search for Optimal Multi-agent Pathfinding
,”
Artif. Intell.
,
219
, pp.
40
66
.
5.
Wagner
,
G.
, and
Choset
,
H.
,
2011
, “
M*: A Complete Multirobot Path Planning Algorithm With Performance Bounds
,”
2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
San Francisco, CA
,
Sept. 25–30
, IEEE, pp.
3260
3267
.
6.
Stern
,
R.
,
Sturtevant
,
N.
,
Felner
,
A.
,
Koenig
,
S.
,
Ma
,
H.
,
Walker
,
T.
,
Li
,
J.
, et al
,
2021
, “
Multi-Agent Pathfinding: Definitions, Variants, and Benchmarks
,”
Proc. Int. Symp. Comb. Search
,
10
(
1
), pp.
151
158
.
7.
Sartoretti
,
G.
,
Kerr
,
J.
,
Shi
,
Y.
,
Wagner
,
G.
,
Kumar
,
T. K. S.
,
Koenig
,
S.
, and
Choset
,
H.
,
2019
, “
PRIMAL: Pathfinding Via Reinforcement and Imitation Multi-agent Learning
,”
IEEE Rob. Autom. Lett.
,
4
(
3
), pp.
2378
2385
.
8.
Li
,
Q.
,
Gama
,
F.
,
Ribeiro
,
A.
, and
Prorok
,
A.
,
2020
, “
Graph Neural Networks for Decentralized Multi-robot Path Planning
,”
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Las Vegas, NV
,
Oct. 25–29
, pp.
11785
11792
.
9.
Barer
,
M.
,
Sharon
,
G.
,
Stern
,
R.
, and
Felner
,
A.
,
2021
, “
Suboptimal Variants of the Conflict-Based Search Algorithm for the Multi-agent Pathfinding Problem
,”
Proc. Int. Symp. Comb. Search
,
5
(
1
), pp.
19
27
.
10.
van den Berg
,
J.
,
Guy
,
S. J.
,
Lin
,
M.
, and
Manocha
,
D.
,
2009
, “
Reciprocal n-Body Collision Avoidance
,”
The 14th International Symposium of Robotics Research ISRR
,
Lucerne, Switzerland
,
Aug. 31–Sept. 3
, pp.
3
19
.
11.
Prorok
,
A.
,
Blumenkamp
,
J.
,
Li
,
Q.
,
Kortvelesy
,
R.
,
Liu
,
Z.
, and
Stump
,
E.
,
2022
, “
The Holy Grail of Multi-Robot Planning: Learning to Generate Online-Scalable Solutions From Offline-Optimal Experts
,”
Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems
,
Auckland, New Zealand
,
May 9–13
,
pp. 1804–1808
.
12.
Lin
,
S.
,
Liu
,
A.
,
Wang
,
J.
, and
Kong
,
X.
,
2022
, “
A Review of Path-Planning Approaches for Multiple Mobile Robots
,”
Machines
,
10
(
9
), p.
773
.
13.
Li
,
Q.
,
Lin
,
W.
,
Liu
,
Z.
, and
Prorok
,
A.
,
2021
, “
Message-Aware Graph Attention Networks for Large-Scale Multi-Robot Path Planning
,”
IEEE Rob. Autom. Lett.
,
6
(
3
), pp.
5533
5540
.
14.
Kool
,
W.
,
van Hoof
,
H.
, and
Welling
,
M.
,
2019
, “
Attention, Learn to Solve Routing Problems!
,”
The 7th International Conference on Learning Representations, ICLR 2019
,
New Orleans, LA
,
May 6–9
.
15.
Paul
,
S.
,
Ghassemi
,
P.
, and
Chowdhury
,
S.
,
2022
, “
Learning Scalable Policies Over Graphs for Multi-Robot Task Allocation Using Capsule Attention Networks
,”
2022 International Conference on Robotics and Automation (ICRA)
,
Philadelphia, PA
,
May 23–27
, pp.
8815
8822
.
16.
Wang
,
Z.
,
Liu
,
C.
, and
Gombolay
,
M.
,
2022
, “
Heterogeneous Graph Attention Networks for Scalable Multi-Robot Scheduling With Temporospatial Constraints
,”
Auton. Rob.
,
46
(
1
), pp.
249
268
.
17.
Paul
,
S.
,
Li
,
W.
,
Smyth
,
B.
,
Chen
,
Y.
,
Gel
,
Y.
, and
Chowdhury
,
S.
,
2023
, “
Efficient Planning of Multi-Robot Collective Transport Using Graph Reinforcement Learning With Higher Order Topological Abstraction
,”
2023 IEEE International Conference on Robotics and Automation (ICRA)
,
London, UK
,
May 29–June 2
, pp.
5779
5785
.
18.
Ma
,
Z.
,
Luo
,
Y.
, and
Ma
,
H.
,
2021
, “
Distributed Heuristic Multi-agent Path Finding With Communication
,”
2021 IEEE International Conference on Robotics and Automation (ICRA)
,
Xi'an, China
,
May 30–June 5
, pp.
8699
8705
.
19.
Zhang
,
S. Q.
,
Lin
,
J.
, and
Zhang
,
Q.
,
2020
, “
Succinct and Robust Multi-agent Communication With Temporal Message Control
,”
Proceedings of the 34th International Conference on Neural Information Processing Systems
,
Vancouver, BC, Canada
,
Dec. 6–12
,
pp. 17271–17282
.
20.
Ding
,
Z.
,
Huang
,
T.
, and
Lu
,
Z.
,
2020
, “
Learning Individually Inferred Communication for Multi-Agent Cooperation
,”
Proceedings of the 34th International Conference on Neural Information Processing Systems
,
Vancouver, BC, Canada
,
Dec. 6–12
, pp.
22069
22079
.
21.
Ma
,
Z.
,
Luo
,
Y.
, and
Pan
,
J.
,
2022
, “
Learning Selective Communication for Multi-Agent Path Finding
,”
IEEE Rob. Autom. Lett.
,
7
(
2
), pp.
1455
1462
.
22.
Sheng
,
J.
,
Wang
,
X.
,
Jin
,
B.
,
Yan
,
J.
,
Li
,
W.
,
Chang
,
T.-H.
,
Wang
,
J.
, and
Zha
,
H.
,
2022
, “
Learning Structured Communication for Multi-agent Reinforcement Learning
,”
Auton. Agent Multi-Agent Syst.
,
36
(
2
), p.
50
.
23.
Chen
,
L.
,
Wang
,
Y.
,
Miao
,
Z.
,
Mo
,
Y.
,
Feng
,
M.
,
Zhou
,
Z.
, and
Wang
,
H.
,
2023
, “
Transformer-Based Imitative Reinforcement Learning for Multi-robot Path Planning
,”
IEEE Trans. Ind. Inform.
,
19
(
10
), pp.
10233
10243
.
24.
Simonyan
,
K.
, and
Zisserman
,
A.
,
2015
, “
Very Deep Convolutional Networks for Large-Scale Image Recognition
,”
3rd International Conference on Learning Representations, ICLR 2015
,
San Diego, CA
,
May 7–9
.
25.
Chung
,
J.
,
Gulcehre
,
C.
,
Cho
,
K.
, and
Bengio
,
Y.
,
2014
, “
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
,”
NIPS 2014 Workshop on Deep Learning
,
Montreal, Quebec, Canada
,
Dec. 8–13
.
26.
Veličković
,
P.
,
Cucurull
,
G.
,
Casanova
,
A.
,
Romero
,
A.
,
Liò
,
P.
, and
Bengio
,
Y.
,
2018
, “
Graph Attention Networks
,”
The 6th International Conference on Learning Representations, ICLR 2018
,
Vancouver, BC, Canada
,
Apr. 30–May 3
.
27.
Vaswani
,
A.
,
Shazeer
,
N.
,
Parmar
,
N.
,
Uszkoreit
,
J.
,
Jones
,
L.
,
Gomez
,
A. N.
,
Kaiser
,
L.
, and
Polosukhin
,
I.
,
2017
, “
Attention Is All You Need
,”
Proceedings of the 31st International Conference on Neural Information Processing Systems
,
Long Beach, CA
,
Dec. 4–9
, pp.
6000
6010
.
28.
Codevilla
,
F.
,
Müller
,
M.
,
López
,
A.
,
Koltun
,
V.
, and
Dosovitskiy
,
A.
,
2018
, “
End-to-End Driving Via Conditional Imitation Learning
,”
2018 IEEE International Conference on Robotics and Automation (ICRA)
,
Brisbane, Australia
,
May 21–25
, pp.
4693
4700
.
You do not currently have access to this content.