Graphical Abstract Figure
Graphical Abstract Figure
Close modal

Abstract

During disasters, such as floods, it is crucial to get real-time ground information for planning rescue and response operations. With the advent of technology, unmanned aerial vehicles (UAVs) are being deployed for real-time path planning to provide support to evacuation teams. However, their dependency on expert human pilots for command and control limits their operational capacity to the line-of-sight range. In this article, we utilize a deep reinforcement learning algorithm to autonomously control multiple UAVs for area coverage. The objective is to identify serviceable paths for safe navigation of waterborne evacuation vehicles (WBVs) to reach critical location(s) during floods. The UAVs are tasked to capture the obstacle-related data and identify shallow water regions for unrestricted motion of the WBV(s). The data gathered by UAVs is used by the minimum expansion A* (MEA*) algorithm for path planning to assist WBV(s). MEA* addresses the node expansion issue with the standard A* algorithm, by pruning the unserviceable nodes/locations based on the captured information, hence expediting the path planning process. The proposed approach, MEA*MADDPG, is compared with other prevalent techniques from the literature over simulated flood environments with moving obstacles. The results highlight the significance of the proposed model as it outperforms other techniques when compared over various performance metrics.

References

1.
UNDRR
,
2024
, “Human Cost of Disasters, an Overview of the Last 20 Years,” https://www.undrr.org/media/48008/download?startDownload=true, Accessed June 25, 2024.
2.
Abdelkader
,
M.
,
Shaqura
,
M.
,
Claudel
,
C. G.
, and
Gueaieb
,
W.
,
2013
, “
A UAV Based System for Real Time Flash Flood Monitoring in Desert Environments Using Lagrangian Microsensors
,”
2013 International Conference on Unmanned Aircraft Systems (ICUAS)
,
Atlanta, GA
,
May 28–31
, pp. 25—34.
3.
Munawar
,
H. S.
,
Hammad
,
A. W.
, and
Waller
,
S. T.
,
2022
, “
Disaster Region Coverage Using Drones: Maximum Area Coverage and Minimum Resource Utilisation
,”
Drones
,
6
(
4
), p.
96
.
4.
Dammen
,
E. B.
,
2022
, “
Reinforcement Learning and Evolutionary Algorithms for Attitude Control, A Comparison for Aerial Vehicles
,” Master's thesis, University of Oslo, Oslo, Norway.
5.
Wang
,
Z.
, and
Hong
,
T.
,
2020
, “
Reinforcement Learning for Building Controls: The Opportunities and Challenges
,”
Appl. Energy.
,
269
(
1
), p.
115036
.
6.
Garg
,
A.
, and
Jha
,
S. S.
,
2023
, “
Real-Time Serviceable Path Planning Using UAVs for Waterborne Vehicle Navigation During Floods
,”
Proceedings of the 2023 6th International Conference on Advances in Robotics (AIR '23)
,
Rupnagar, Punjab, India
,
July 5–8
.
7.
Rudnick-Cohen
,
E.
,
Herrmann
,
J. W.
, and
Azarm
,
S.
,
2016
, “
Risk-Based Path Planning Optimization Methods for Unmanned Aerial Vehicles Over Inhabited Areas
,”
ASME J. Comput. Inf. Sci. Eng.
,
16
(
2
), p.
021004
.
8.
Farid
,
G.
,
Cocuzza
,
S.
,
Younas
,
T.
,
Razzaqi
,
A. A.
,
Wattoo
,
W. A.
,
Cannella
,
F.
, and
Mo
,
H.
,
2022
, “
Modified A-star (A*) Approach to Plan the Motion of a Quadrotor UAV in Three-Dimensional Obstacle-Cluttered Environment
,”
Appl. Sci.
,
12
(
12
), p.
5791
.
9.
Zhang
,
B.
,
Li
,
G.
,
Zheng
,
Q.
,
Bai
,
X.
,
Ding
,
Y.
, and
Khan
,
A.
,
2022
, “
Path Planning for Wheeled Mobile Robot in Partially Known Uneven Terrain
,”
Sensors
,
22
(
14
), p.
5217
.
10.
Yang
,
W.
,
Wang
,
G.
, and
Shen
,
Y.
,
2020
, “
Perception-Aware Path Finding and Following of Snake Robot in Unknown Environment
,”
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Las Vegas, NV
,
Oct. 25–29
, pp.
5925
5930
.
11.
Puente-Castro
,
A.
,
Rivero
,
D.
,
Pedrosa
,
E.
,
Pereira
,
A.
,
Lau
,
N.
, and
Fernandez-Blanco
,
E.
,
2023
, “
Q-learning Based System for Path Planning With Unmanned Aerial Vehicles Swarms in Obstacle Environments
,”
Expert. Syst. Appl.
,
235
(
1
), p.
121240
.
12.
Bashir
,
N.
,
Boudjit
,
S.
,
Dauphin
,
G.
, and
Zeadally
,
S.
,
2023
, “
An Obstacle Avoidance Approach for UAV Path Planning
,”
Simul. Modell. Practice Theory
,
129
(
1
), p.
102815
.
13.
Puente-Castro
,
A.
,
Rivero
,
D.
,
Pazos
,
A.
, and
Fernandez-Blanco
,
E.
,
2022
, “
UAV Swarm Path Planning With Reinforcement Learning for Field Prospecting
,”
Appl. Intell.
,
52
(
12
), pp.
14101
14118
.
14.
Yan
,
C.
,
Xiang
,
X.
, and
Wang
,
C.
,
2019
, “
Towards Real-Time Path Planning Through Deep Reinforcement Learning for a UAV in Dynamic Environments
,”
J. Intell. Rob. Syst.
,
98
(
2
), pp.
297
309
.
15.
Ali
,
Z. A.
, and
Zhangang
,
H.
,
2021
, “
Multi-unmanned Aerial Vehicle Swarm Formation Control Using Hybrid Strategy
,”
Trans. Inst. Measurem. Control
,
43
(
12
), pp.
2689
2701
.
16.
Kaushik
,
P.
,
Garg
,
A.
, and
Jha
,
S. S.
,
2021
, “
On Learning Multi-UAV Policy for Multi-Object Tracking and Formation Control
,”
2021 IEEE 18th India Council International Conference (INDICON)
,
Guwahati, Assam, India
,
Dec. 19–21
, pp.
1
6
.
17.
Liu
,
H.
,
Peng
,
F.
,
Modares
,
H.
, and
Kiumarsi
,
B.
,
2021
, “
Heterogeneous Formation Control of Multiple Rotorcrafts With Unknown Dynamics by Reinforcement Learning
,”
Inf. Sci.
,
558
(
1
), pp.
194
207
.
18.
Zhao
,
W.
,
Liu
,
H.
,
Wan
,
Y.
, and
Lin
,
Z.
,
2022
, “
Data-Driven Formation Control for Multiple Heterogeneous Vehicles in Air-Ground Coordination
,”
IEEE Trans. Control Netw. Syst.
,
9
(
4
), pp.
1851
1862
.
19.
Papoutsellis
,
C. E.
,
2015
, “
Numerical Simulation of Non-linear Water Waves Over Variable Bathymetry
,”
Procedia Comput. Sci.
,
66
(
1
), pp.
174
183
.
20.
Zeng
,
Y.
,
Xu
,
J.
, and
Zhang
,
R.
,
2019
, “
Energy Minimization for Wireless Communication With Rotary-Wing UAV
,”
IEEE Trans. Wireless Commun.
,
18
(
4
), pp.
2329
2345
.
21.
Lowe
,
R.
,
Wu
,
Y.
,
Tamar
,
A.
,
Harb
,
J.
,
Abbeel
,
P.
, and
Mordatch
,
I.
,
2017
, “
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
,”
Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17)
,
Long Beach, CA
,
Dec. 4–9
, Curran Associates Inc., pp.
6382
6393
.
22.
Braun
,
J. a.
,
Brito
,
T.
,
Lima
,
J.
,
Costa
,
P.
,
Costa
,
P.
, and
Nakano
,
A.
,
2019
, “
A Comparison of A* and RRT* Algorithms With Dynamic and Real Time Constraint Scenarios for Mobile Robots
,”
Proceedings of the 9th International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH 2019)
,
Prague, Czech Republic
,
July 29–31
, pp.
398
405
.
23.
Hu
,
C.
, and
Jin
,
Y.
,
2023
, “
Long-Range Risk-Aware Path Planning for Autonomous Ships in Complex and Dynamic Environments
,”
ASME J. Comput. Inf. Sci. Eng.
,
23
(
4
), p.
041007
.
24.
Felner
,
A.
,
Goldenberg
,
M.
,
Sharon
,
G.
,
Stern
,
R.
,
Beja
,
T.
,
Sturtevant
,
N.
,
Schaeffer
,
J.
, and
Holte
,
R. C.
,
2012
, “
Partial-Expansion A* With Selective Node Generation
,”
Twenty-Sixth AAAI Conference on Artificial Intelligence
,
Toronto, Ontario, Canada
,
July 22–26
, AAAI Press, pp.
471
477
.
25.
Pham
,
H. X.
,
La
,
H. M.
,
Feil-Seifer
,
D.
, and
Deans
,
M.
,
2017
, “
A Distributed Control Framework for a Team of Unmanned Aerial Vehicles for Dynamic Wildfire Tracking
,”
2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Vancouver, BC, Canada
,
Sept. 24–28
, pp.
6648
6653
.
26.
Nasir
,
J.
,
Islam
,
F.
,
Malik
,
U. A.
,
Ayaz
,
Y.
,
Hasan
,
O.
,
Khan
,
M.
, and
Muhammad
,
M.
,
2013
, “
RRT*-smart: A Rapid Convergence Implementation of RRT*’
,”
Int. J. Adv. Rob. Syst.
,
10
(
7
), p.
299
.
27.
Liu
,
H.
, and
Abbeel
,
P.
,
2021
, “
Behavior From the Void: Unsupervised Active Pre-training
,”
35th Conference on Neural Information Processing Systems (NeurIPS 2021)
,
Virtual-only Conference
,
Dec. 6–14
.
You do not currently have access to this content.