Graphical Abstract Figure

System Architecture for Training and Evaluating the RL-Based Chaser Drone Policy with Gazebo, ROS, YOLOv5, DDPG, and Ardupilot

Graphical Abstract Figure

System Architecture for Training and Evaluating the RL-Based Chaser Drone Policy with Gazebo, ROS, YOLOv5, DDPG, and Ardupilot

Close modal

Abstract

Unmanned aerial vehicles (UAVs) are fast becoming a low-cost, affordable tool for various security and surveillance tasks. It has led to the use of UAVs (drones) for unlawful activities such as spying or infringing on restricted or private air spaces. This rogue use of drone technology makes it challenging for security agencies to maintain the safety of many critical infrastructures. Additionally, because of the drones’ varied low-cost design and agility, it has become challenging to identify and track them using conventional radar systems. This paper proposes a deep reinforcement learning-based approach for identifying and tracking an intruder drone using a chaser drone. Our proposed solution employs computer vision techniques interleaved with a deep reinforcement learning control for tracking the intruder drone within the chaser’s field of view. The complete end-to-end system has been implemented using robot operating system and Gazebo, with an Ardupilot-based flight controller for flight stabilization and maneuverability. The proposed approach has been evaluated on multiple dynamic scenarios of intruders’ trajectories and compared with a proportional-integral-derivative-based controller. The results show that the deep reinforcement learning policy achieves a tracking accuracy of 85%. The intruder localization module is able to localize drones in 98.5% of the frames. Furthermore, the learned policy can track the intruder even when there is a change in the speed or orientation of the intruder drone.

References

1.
Zeng
,
Y.
,
Lyu
,
J.
, and
Zhang
,
R.
,
2018
, “
Cellular-Connected UAV: Potential, Challenges, and Promising Technologies
,”
IEEE Wireless Commun.
,
26
(
1
), pp.
120
127
.
2.
Garg
,
A.
, and
Jha
,
S. S.
,
2023
, “
Decentralized Critical Area Coverage Using Multi-UAV System With Guided Explorations During Floods
,”
CASE
, pp.
1
6
.
3.
Basak
,
S.
,
Rajendran
,
S.
,
Pollin
,
S.
, and
Scheers
,
B.
,
2021
, “
Combined RF-Based Drone Detection and Classification
,”
IEEE Trans. Cognit. Commun. Networking
,
8
(
1
), pp.
111
120
.
4.
De Souza
,
C. N.
,
2021
, “
Decentralized Multi-agent Pursuit Using Deep Reinforcement Learning
,”
IEEE Rob. Autom. Lett.
,
6
(
3
), pp.
4552
4559
.
5.
Kainth
,
S.
,
Sahoo
,
S.
,
Pal
,
R.
, and
Jha
,
S. S.
,
2023
, “
Chasing the Intruder: A Reinforcement Learning Approach for Tracking Unidentified Drones
,”
Advances In Robotics
,
Ropar, India
,
Aug. 29
, ACM, pp.
1
6
.
6.
Opromolla
,
R.
,
Inchingolo
,
G.
, and
Fasano
,
G.
,
2019
, “
Airborne Visual Detection and Tracking of Cooperative UAVs Exploiting Deep Learning
,”
Sensors
,
19
(
19
), p.
4332
.
7.
Unlu
,
E.
,
Zenou
,
E.
,
Riviere
,
N.
, and
Dupouy
,
P.-E.
,
2019
, “
Deep Learning-Based Strategies for the Detection and Tracking of Drones Using Several Cameras
,”
IPSJ Trans. Comput. Vis. Appl.
,
11
(
1
), pp.
1
13
.
8.
Zhang
,
X.
, and
Kusrini
,
K.
,
2021
, “
Autonomous Long-Range Drone Detection System for Critical Infrastructure Safety
,”
Multimedia Tools Appl.
,
80
(
15
), pp.
23723
23743
.
9.
Barisic
,
A.
,
Petric
,
F.
, and
Bogdan
,
S.
,
2021
, “Brain Over Brawn: Using a Stereo Camera to Detect, Track, and Intercept a faster UAV by Reconstructing the Intruder’s Trajectory,” preprint arXiv:2107.00962.
10.
Rozantsev
,
A.
,
Lepetit
,
V.
, and
Fua
,
P.
,
2015
, “
Flying Objects Detection From a Single Moving Camera
,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, June 7–12, pp.
4128
4136
.
11.
Wei
,
Y.
,
Hong
,
T.
, and
Kadoch
,
M.
,
2020
, “
Improved Kalman Filter Variants for UAV Tracking With Radar Motion Models
,”
Electronics
,
9
(
5
), p.
768
.
12.
Pham
,
H. X. L.
,
2018
, “
Reinforcement Learning for Autonomous UAV Navigation Using Function Approximation
,” 2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR),
Philadelphia, PA, Aug. 6–8, IEEE
, pp.
1
6
.
13.
Akhloufi
,
M. A.
,
Arola
,
S.
, and
Bonnet
,
A.
,
2019
, “
Drones Chasing Drones: Reinforcement Learning and Deep Search Area Proposal
,”
Drones
,
3
(
3
), p.
58
.
14.
Fernandes
,
A. B.
,
2019
, “
Drone, Aircraft and Bird Identification in Video Images Using Object Tracking and Residual Neural Networks
,”
2019 11th International Conference on Electronics, Computers and Artificial Intelligence (ECAI)
,
Pitesti, Romania
,
June 27–29
,
IEEE
, pp.
1
6
.
15.
Hu
,
Y.
,
Wu
,
X.
,
Zheng
,
G.
, and
Liu
,
X.
,
2019
, “
Object Detection of UAV for Anti-UAV Based on Improved Yolo V3
,” 2019 Chinese Control Conference (CCC),
IEEE
, pp.
8386
8390
.
16.
Gao
,
J. B.
,
2024
, “
Analysis of Various Machine Learning Algorithms for Using Drone Images in Livestock Farms
,”
Agriculture
,
14
(
4
), p.
522
.
17.
Zeng
,
L.
,
Chen
,
H.
,
Feng
,
D.
,
Zhang
,
X.
, and
Chen
,
X.
,
2024
, “
A3D: Adaptive, Accurate, and Autonomous Navigation for Edge-Assisted Drones
,”
IEEE/ACM Trans. Networking
,
32
(
1
), pp.
713
728
.
18.
Nepal
,
U.
,
2022
, “
Comparing YOLOv3, YOLOv4 and YOLOv5 for Autonomous Landing Spot Detection in Faulty UAVs
,”
Sensors
,
22
(
2
), p.
464
.
19.
Silver
,
D.
,
Lever
,
G.
,
Heess
,
N.
,
Degris
,
T.
,
Wierstra
,
D.
, and
Riedmiller
,
M.
,
2014
, “
Deterministic Policy Gradient Algorithms
,”
Proceedings of Machine Learning Research
, pp.
387
395
. https://api.semanticscholar.org/CorpusID:13928442
You do not currently have access to this content.