The control of networked multivehicle systems designed to perform complex coordinated tasks is currently an important and challenging field of research. This paper addresses a cooperative search problem where a team of uninhabited aerial vehicles (UAVs) seeks to find targets of interest in an uncertain environment. We present a practical framework for online planning and control of a group of UAVs for cooperative search based on two interdependent tasks: (i) incrementally updating “cognitive maps” used as the representation of the environment through new sensor readings; (ii) continuously planning the path for each vehicle based on the information obtained through the search. We formulate the cooperative search problem and develop a decentralized strategy based on an opportunistic cooperative learning method, where the emergent coordination among vehicles is enabled by letting each vehicle consider other vehicles’ actions in its path planning procedure. By using the developed strategy, physically feasible paths for the vehicles to follow are generated, where constraints on aerial vehicles, including physical maneuverabilities, are considered and the dynamic nature of the environment is taken into account. We also present some mathematical analysis of the developed search strategy. Our analysis shows that this strategy guarantees a complete search of the environment and is robust to a partial loss of UAVs. A lower bound on the search time for any strategy and a relaxed upper bound for the proposed strategy are given. Simulation results are used to illustrate the effectiveness of the proposed strategy.

1.
Schoenwald
,
D. A.
, 2000, “
AUVs: In Space, Air, Water, and on the Ground
,”
IEEE Control Syst. Mag.
0272-1708,
20
(
6
), pp.
15
18
.
2.
Office of The Under Secretary of Defense for Acquisition Technology and Logistics
Washington DC, 2002, “
Unmanned Aerial Vehicles Roadmap 2002–2027
,” Office of the Secretary of Defense Technical Report A809414;
3.
Chandler
,
P.
,
Rasmussen
,
S.
, and
Pachter
,
M.
, 2000, “
UAV Cooperative Path Planning
,”
Proceedings of AIAA Guidance, Navigation, and Control Conference and Exhibit
,
Denver, CO
, pp.
1255
1265
.
4.
Chandler
,
P.
, and
Pachter
,
M.
, 2001, “
Hierarchical Control for Autonomous Teams
,”
Proceedings of AIAA Guidance, Navigation, and Control Conference and Exhibit
,
Monterey, CA
, pp.
632
642
.
5.
Pachter
,
M.
, and
Chandler
,
P.
, 1998 “
Challenges of Autonomous Control
,”
IEEE Control Syst. Mag.
0272-1708,
18
(
4
), pp.
92
97
.
6.
Nygard
,
K.
,
Chandler
,
P.
, and
Pachter
,
M.
, 2001, “
Dynamic Network Optimization Models for Air Vehicle Resource Allocation
,”
Proceedings of the 2001 American Control Conference
,
Arlington, VA
, Vol.
3
, pp.
1853
1856
.
7.
Jin
,
Y.
,
Minai
,
A. A.
, and
Polycarpou
,
M. M.
, 2003, “
Cooperative Real-Time Search and Task Allocation in UAV Teams
,”
Proceedings of the 42nd IEEE Conference on Decision and Control
,
Maui, HI
, Vol.
1
, pp.
7
12
.
8.
Chandler
,
P.
, 2001, “
UAV Cooperative Classification
,”
Workshop on Cooperative Control and Optimization
,
Kluwer Academic
,
Dordrecht
, pp.
1
20
.
9.
Bellingham
,
J. S.
,
Tillerson
,
M.
,
Alighanbari
,
M.
, and
How
,
J. P.
, 2002, “
Cooperative Path Planning for Multiple UAVs in Dynamic and Uncertain Environment
,”
Proceedings of the 41st IEEE Conference on Decision and Control
,
Las Vegas, NV
, Vol.
9
, pp.
2816
2822
.
10.
Yang
,
Y.
,
Polycarpou
,
M.
, and
Minai
,
A.
, 2004, “
Decentralized Cooperative Search by Networked Multi-UAVs in Uncertain Environment
,”
Proceedings of the 2004 American Control Conference
,
Boston, MI
, Vol.
6
, pp.
5558
5563
.
11.
Flint
,
M.
,
Polycarpou
,
M.
, and
Fernandez
,
E.
, 2003, “
Stochastic Models of a Cooperative Autonomous UAV Search Problem
,”
Military Operations Research Journal
,
8
(
4
), pp.
13
32
.
12.
Vincent
,
P.
, and
Rubin
,
I.
, 2004, “
A Framework and Analysis for Cooperative Search Using UAV Swarms
,”
Proceedings of the 2004 ACM Symposium of Applied Computing
,
Nicosia, Cyprus
, Vol.
1
, pp.
79
86
.
13.
McLain
,
T.
,
Chandler
,
P.
,
Rasmussen
,
S.
, and
Pachter
,
M.
, 2001, “
Cooperative Control of UAV Rendezvous
,”
Proceedings of the 2001 American Control Conference
,
Arlington, VA
, Vol.
3
, pp.
2309
2314
.
14.
Tanner
,
H.
,
Jadbabaie
,
A.
, and
Pappas
,
G. J.
, 2003, “
Stable Flocking of Mobile Agents, Part I: Fixed Topology
,”
Proceedings of the 42nd IEEE Conference on Decision and Control
,
Maui, HI
, Vol.
2
, pp.
2010
2015
.
15.
Tanner
,
H.
,
Jadbabaie
,
A.
, and
Pappas
,
G. J.
, 2003, “
Stable Flocking of Mobile Agents, Part II: Dynamic Topology
,”
Proceedings of the 42nd IEEE Conference on Decision and Control
,
Maui, HI
, Vol.
2
, pp.
2016
2021
.
16.
Polycarpou
,
M.
,
Yang
,
Y.
,
Liu
,
L.
, and
Passino
,
K.
, 2003, “
Cooperative Control Design for Uninhabited Air Vehicles
,”
Cooperative Control: Models, Applications and Algorithms
,
Kluwer Academic
,
Dordrecht
, pp.
283
321
.
17.
Polycarpou
,
M.
,
Yang
,
Y.
, and
Passino
,
K.
, 2001, “
A Cooperative Search Framework for Distributed Agents
,”
Proceedings of the 2001 IEEE International Symposium on Intelligent Control
,
Mexico City, Mexico
, pp.
1
6
.
18.
Tan
,
M.
, 1993, “
Multi-Agent Reinforcement Learning: Independent Versus Cooperative Agents
,”
Proceedings of the Tenth International Conference on Machine Learning
, pp.
330
337
.
19.
Koopman
,
B.
, 1980,
Search and Screening: General Principles With Historical Application
,
Pergamon
,
New York
.
20.
Stone
,
L.
, 1995,
Theory of Optimal Search
,
Academic
,
New York
.
21.
Eagle
,
J.
, and
Yee
,
J.
, 1990, “
An Optimal Branch-and-Bound Procedure for the Constrained Path Moving Target Search Problem
,”
Oper. Res.
0030-364X,
38
(
1
), pp.
110
114
.
22.
Hohzaki
,
R.
, and
Iida
,
K.
, 1995, “
Path Constrained Search Problem With Reward Criterion
,”
J. Oper. Res. Soc. Jpn.
0453-4514,
38
(
2
), pp.
254
264
.
23.
Hohzaki
,
R.
, and
Iida
,
K.
, 1995, “
An Optimal Search Plan for a Moving Target When a Search Path is Given
,”
Math. Japonica
0025-5513,
41
(
1
), pp.
175
184
.
24.
Benkoski
,
S.
,
Monticino
,
M.
, and
Weisinger
,
J.
, 1991, “
A Survey of the Search Theory Literature
,”
Naval Res. Logistics Quart.
0894-069X,
38
, pp.
469
494
.
25.
Richardson
,
H.
, 1987, in
Search Theory: Some Recent Developments
,
D.
Chudnovsky
and
G.
Chudnovsky
, eds.,
Marcel Dekker
,
New York
, pp.
1
12
.
26.
Chandler
,
P.
,
Pachter
,
M.
, and
Rasmussen
,
S.
, 2001, “
UAV Cooperative Control
,”
Proceedings of the 2001 American Control Conference
,
Arlington, VA
, Vol.
1
, pp.
50
55
.
27.
Bellingham
,
J. S.
,
Tillerson
,
M.
,
Richards
,
A. G.
, and
How
,
J. P.
, 2001, “
Multi-Task Assignment and Path Planning for Cooperating UAVs
,”
Conference on Cooperative Control and Optimization
,
Kluwer Academic
,
Dordrecht
, pp.
1
19
.
28.
Jin
,
Y.
,
Minai
,
A. A.
, and
Polycarpou
,
M. M.
, 2004, “
Balancing Search and Target Response in Cooperative UAV Team
,”
Proceedings of the 43rd IEEE Conference on Decision and Control
,
Atlantis, Paradise Island, Bahamas
, Vol.
3
, pp.
2923
2928
.
29.
Murphey
,
R. A.
, 1999, “
An Approximate Algorithm for a Weapon Target Assignment Stochastic Program
,”
Approximate and Complexity in Numerical Optimization: Continuous and Discrete Problem
,
Kluwer Academic
,
Dordrecht
.
30.
Burgard
,
W.
,
Fox
,
D.
,
Moors
,
M.
,
Simmons
,
R.
, and
Thrun
,
S.
, 2000, “
Collaborative Multi-Robot Exploration
,”
Proceedings of International Conference on Robotics and Automation
, Vol.
1
, pp.
476
481
.
31.
Choset
,
H.
, and
Pignon
,
P.
, 1997, “
Coverage Path Planning: The Boustrophedon Cellular Decomposition
,”
International Conference on Field and Service Robotics
,
Canberra, Australia
.
32.
Rekleitis
,
I.
,
Dudek
,
G.
, and
Milios
,
E.
, 1998, “
Accurate Mapping of an Unknown World and Online Landmark Positioning
,”
Proceedings of Vision Interface
, pp.
455
461
.
33.
Svennebring
,
J.
, and
Koenig
,
S.
, 2004, “
Building Terrain-Covering Ant Robots: A Feasibility Study
,”
Auton. Rob.
0929-5593,
16
(
3
), pp.
313
332
.
34.
Wagner
,
I. A.
,
Lindenbaum
,
M.
, and
Bruckstein
,
A. M.
, 2001, “
MAC Versus PC: Determinism and Randomness as Complementary Approaches to Robotic Exploration of Continuous Unknown Domains
,”
Int. J. Robot. Res.
0278-3649,
19
(
1
), pp.
313
332
.
35.
Yang
,
S.
, and
Luo
,
C.
, 2004, “
A Neural Network Approach to Complete Coverage Path Planning
,”
IEEE Trans. Syst., Man, Cybern., Part B: Cybern.
1083-4419,
34
(
1
), pp.
718
725
.
36.
Choset
,
H.
, 2001, “
Coverage for Robotics—A Survey of Recent Results
,”
Ann. Math. Artif. Intell.
1012-2443,
31
(1–4), pp.
113
126
.
37.
Yang
,
Y.
,
Polycarpou
,
M.
, and
Minai
,
A.
, 2002, “
Opportunistically Cooperative Neural Learning in Mobile Agents
,”
Proceedings of the 2002 International Joint Conference on Neural Networks
,
Honolulu, HI
, Vol.
3
, pp.
2638
2643
.
38.
Yang
,
Y.
,
Minai
,
A.
, and
Polycarpou
,
M.
, 2002, “
Decentralized Cooperative Search in UAV’s Using Opportunistic Learning
,”
Proceedings of AIAA Guidance, Navigation, and Control Conference and Exhibit
,
Monterey, CA
, Paper No. AIAA-2002-4590.
39.
Yang
,
Y.
,
Minai
,
A.
, and
Polycarpou
,
M.
, 2002, “
Analysis of Opportunistic Method for Cooperative Search by Mobile Agents
,”
Proceedings of the 41st IEEE Conference on Decision and Control
,
Las Vegas, NV
, Vol.
1
, pp.
576
577
.
40.
Yang
,
Y.
,
Polycarpou
,
M.
, and
Minai
,
A.
, 2005, “
Evidential Map-Building Approaches for Multi-UAV Cooperative Search
,”
Proceedings of the 2005 American Control Conference
,
Portland, OR
, pp.
116
121
.
41.
Flint
,
M.
,
Fernandez
,
E.
, and
Polycarpou
,
M.
, 2003, “
Cooperative Control for UAVs Searching Risky Environments for Targets
,”
Proceedings of the 42nd IEEE Conference on Decision and Control
,
Maui, HI
, Vol.
4
, pp.
3567
-–
3572
.
42.
Flint
,
M.
,
Polycarpou
,
M.
, and
Fernandez
,
E.
, 2002, “
Cooperative Control for Multiple Autonomous UAV’s Searching for Targets
,”
Proceedings of the 41st IEEE Conference on Decision and Control
,
Las Vegas, NV
, Vol.
3
, pp.
2823
2828
.
43.
Flint
,
M.
,
Polycarpou
,
M.
, and
Fernandez
,
E.
, 2002, “
Cooperative Path-Planning for Autonomous Vehicles Using Dynamic Programming
,”
Proceedings of the IFAC 15th Triennial World Congress
,
Barcelona, Spain
, pp.
1694
1699
.
44.
Sujit
,
P. B.
, and
Ghose
,
D.
, 2004, “
Multiple Agent Search of an Unknown Environment Using Game Theoretical Models
,”
IEEE Trans. Aerosp. Electron. Syst.
0018-9251,
40
(
1–4
), pp.
491
508
.
45.
Sujit
,
P. B.
, and
Ghose
,
D.
, 2003, “
Optimal Uncertainty Reduction Search Using the k-Shortest Path Algorithm
,”
Proceedings of the 2003 American Control Conference
,
Denver, CO
, Vol.
3
, pp.
3269
3274
.
46.
Enns
,
D.
,
Bugajski
,
D.
, and
Pratt
,
S.
, 2002, “
Guidance and Control for Cooperative Search
,”
Proceedings of the 2002 American Control Conference
,
Anchorage, AK
, Vol.
3
, pp.
1923
1929
.
47.
Sujit
,
P. B.
, and
Ghose
,
D.
, 2004, “
Multiple Agent Search of an Unknown Environment Using Game Theoretical Models
,”
Proceedings of the 2004 American Control Conference
,
Boston, MI
, Vol.
6
, pp.
5564
5569
.
48.
Baum
,
M. L.
, and
Passino
,
K. M.
, 2002, “
A Search-Theoretic Approach to Cooperative Control for Uninhabited Air Vehicles
,”
Proceedings of AIAA Guidance, Navigation, and Control Conference and Exhibit
,
Monterey, CA
, Paper No. AIAA-2002-4589.
49.
Zhang
,
C.
, and
Ordonez
,
R.
, 2003, “
Decentralized Adaptive Coordination and Control of Uninhabited Autonomous Vehicles Via Surrogate Optimization
,”
Proceedings of the 2003 American Control Conference
, Vol.
3
, pp.
2205
2210
.
50.
Ablavsky
,
V.
, and
Snorrason
,
M.
, 2000, “
Optimal Search for a Moving Target: A Geometric Approach
,”
Proceedings of AIAA Guidance, Navigation, and Control Conference and Exhibit
,
Denver, CO
, pp.
14
17
.
51.
Tirumalai
,
A. P.
,
Schunck
,
B. G.
, and
Jain
,
R. C.
, 1995, “
Evidential Reasoning for Building Environment Maps
,”
IEEE Trans. Syst. Man Cybern.
0018-9472,
25
(
1
), pp.
10
20
.
52.
Bertsekas
,
D. P.
, 1995,
Dynamic Programming and Optimal Control
,
Athena Scientific
,
MA
, Vol.
1
.
53.
Dubins
,
L.
, 1957, “
On Curves of Minimal Length With a Constraint on Average Curvature and With Prescribed
,”
Am. J. Math.
0002-9327,
79
(
3
), pp.
497
516
.
54.
Wagner
,
I. A.
,
Lindenbaum
,
M.
, and
Bruckstein
,
A. M.
, 1999, “
Distributed Covering by Ant-Robots Using Evaporating Traces
,”
IEEE Trans. Rob. Autom.
1042-296X,
15
(
5
), pp.
918
933
.
55.
Watkins
,
C. J.
, 1989, “
Learning With Delayed Rewards
,” Ph.D thesis, University of Cambridge.
56.
Pagac
,
D.
,
Nebot
,
E. M.
, and
Durrant-Whyte
,
H.
, 1998, “
An Evidential Approach to Map-Building for Autonomous Vehicles
,”
IEEE Trans. Rob. Autom.
1042-296X,
14
(
v
), pp.
623
629
.
You do not currently have access to this content.