This tutorial paper presents the expositions of stochastic optimal feedback control theory and Bayesian spatiotemporal models in the context of robotics applications. The presented material is self-contained so that readers can grasp the most important concepts and acquire knowledge needed to jump-start their research. To facilitate this, we provide a series of educational examples from robotics and mobile sensor networks.
Issue Section:
Review Article
References
1.
Bertsekas
, D. P.
, 1995
, Dynamic Programming and Optimal Control
, Vol. 1
, Athena Scientific
, Belmont
, MA.2.
Bryson
, A. E.
, and Ho
, Y.-C.
, 1975
, Applied Optimal Control: Optimization, Estimation, and Control
, Taylor & Francis
, New York.3.
Pontryagin
, L. S.
, 1962
, The Mathematical Theory of Optimal Processes
, Vol. 4
, Interscience Publishers, New York.
4.
Young
, J.
, and Zhou
, X. Y.
, 1999
, Stochastic Controls: Hamiltonian Systems and HJB Equations
, Vol. 43
, Springer
, New York.5.
Kushner
, H. J.
, and Dupuis
, P.
, 2001
, Numerical Methods for Stochastic Control Problems in Continuous Time
, Vol. 24
, Springer
, New York.10.1007/978-1-4613-0007-66.
Stengel
, R. F.
, 1986
, Optimal Control and Estimation
, Dover
, New York.7.
Oksendal
, B.
, 2003
, Stochastic Differential Equations: An Introduction With Applications
, Springer Verlag
, New York
.8.
Davis
, M. H.
, 1993
, Markov Models and Optimization
, Vol. 49
, Chapman & Hall/CRC, Boca Raton, FL
.9.
Koutsoukos
, X. D.
, 2004
, “Optimal Control of Stochastic Hybrid Systems Based on Locally Consistent Markov Decision Processes
,” Proceedings of the 2005 IEEE International Symposium on Intelligent Control, Limassol, Cyprus, pp. 435–440.10.
Fleming
, W. H.
, and Soner
, H. M.
, 2006
, Controlled Markov Processes and Viscosity Solutions
, Vol. 25
, Springer
, New York.
11.
Engel
, Y.
, Mannor
, S.
, and Meir
, R.
, 2005
, “Reinforcement Learning With Gaussian Processes
,” Proceedings of the 22nd International Conference on Machine Learning
, University of Bonn, Germany, ACM, pp. 201
–208
.10.1145/1102351.110237712.
Engel
, Y.
, Mannor
, S.
, and Meir
, R.
, 2003
, “Bayes Meets Bellman: The Gaussian Process Approach to Temporal Difference Learning
,” Proceedings of the 20th International Conference on Machine Learning (ICML-2003), Washington, DC, pp. 154
–161
.13.
Deisenroth
, M. P.
, Rasmussen
, C. E.
, and Peters
, J.
, 2009
, “Gaussian Process Dynamic Programming
,” Neurocomputing
, 72
(7
), pp. 1508
–1524
.10.1016/j.neucom.2008.12.01914.
Leonard
, N. E.
, Paley
, D. A.
, Lekien
, F.
, Sepulchre
, R.
, Fratantoni
, D. M.
, and Davis
, R.
, 2007
, “Collective Motion, Sensor Networks, and Ocean Sampling
,” Proc. IEEE
, 95
(1
), pp. 48
–74
.10.1109/JPROC.2006.88729515.
Lynch
, K. M.
, Schwartz
, I. B.
, Yang
, P.
, and Freeman
, R. A.
, 2008
, “Decentralized Environmental Modeling by Mobile Sensor Networks
,” IEEE Trans. Rob.
, 24
(3
), pp. 710
–724
.10.1109/TRO.2008.92156716.
Choi
, J.
, Oh
, S.
, and Horowitz
, R.
, 2009
, “Distributed Learning and Cooperative Control for Multi-Agent Systems
,” Automatica
, 45
(12
), pp. 2802
–2814
.10.1016/j.automatica.2009.09.02517.
Xu
, Y.
, Choi
, J.
, and Oh
, S.
, 2011
, “Mobile Sensor Network Navigation Using Gaussian Processes With Truncated Observations
,” IEEE Trans. Rob.
, 27
(6
), pp. 1118
–1131
.10.1109/TRO.2011.216276618.
Xu
, Y.
, Choi
, J.
, Dass
, S.
, and Maiti
, T.
, 2013
, “Efficient Bayesian Spatial Prediction With Mobile Sensor Networks Using Gaussian Markov Random Fields
,” Automatica
, 49
(12
), pp. 3520
–3530
.10.1016/j.automatica.2013.09.00819.
Le Ny
, J.
, and Pappas
, G.
, 2013
, “Adaptive Deployment of Mobile Robotic Networks
,” IEEE Trans. Autom. Control
, 58
(3
), pp. 654
–666
.10.1109/TAC.2012.221551220.
Jadaliha
, M.
, and Choi
, J.
, 2013
, “Environmental Monitoring Using Autonomous Aquatic Robots: Sampling Algorithms and Experiments
,” IEEE Trans. Control Syst. Technol.
, 21
(3
), pp. 899
–905
.10.1109/TCST.2012.219007021.
Cao
, Y.
, Yu
, W.
, Ren
, W.
, and Chen
, G.
, 2013
, “An Overview of Recent Progress in the Study of Distributed Multi-Agent Coordination
,” IEEE Trans. Ind. Inf.
, 9
(1
), pp. 427
–438
.10.1109/TII.2012.221906122.
Kalnay
, E.
, 2003
, Atmospheric Modeling, Data Assimilation, and Predictability
, Cambridge University
, New York.10.1017/CBO978051180227023.
Cortés
, J.
, 2009
, “Distributed Kriged Kalman Filter for Spatial Estimation
,” IEEE Trans. Autom. Control
, 54
(12
), pp. 2816
–2827
.10.1109/TAC.2009.203419224.
Graham
, R.
, and Cortés
, J.
, 2009
, “Cooperative Adaptive Sampling of Random Fields With Partially Known Co-Variance
,” Int. J. Rob. Nonlinear Control
, 22
(5), pp. 504
–534
.25.
Graham
, R.
, and Cortés
, J.
, 2012
, “Adaptive Information Collection by Robotic Sensor Networks for Spatial Estimation
,” IEEE Trans. Autom. Control
, 57
(6
), pp. 1404
–1419
.10.1109/TAC.2011.217833226.
Xu
, Y.
, Choi
, J.
, Dass
, S.
, and Maiti
, T.
, 2012
, “Sequential Bayesian Prediction and Adaptive Sampling Algorithms for Mobile Sensor Networks
,” IEEE Trans. Autom. Control
, 57
(8
), pp. 2078
–2084
.10.1109/TAC.2012.219018127.
Xu
, Y.
, and Choi
, J.
, 2012
, “Spatial Prediction With Mobile Sensor Networks Using Gaussian Processes With Built-In Gaussian Markov Random Fields
,” Automatica
, 48
(8
), pp. 1735
–1740
.10.1016/j.automatica.2012.05.02928.
Xu
, Y.
, and Choi
, J.
, 2012
, “Stochastic Adaptive Sampling for Mobile Sensor Networks Using Kernel Regression
,” Int. J. Control Autom. Syst.
, 10
(4
), pp. 778
–786
.10.1007/s12555-012-0414-529.
Varagnolo
, D.
, Pillonetto
, G.
, and Schenato
, L.
, 2012
, “Distributed Parametric and Nonparametric Regression With On-Line Performance Bounds Computation
,” Automatica
, 48
(10
), pp. 2468
–2481
.10.1016/j.automatica.2012.06.08030.
Samson
, C.
, and Ait-Abderrahim
, K.
, 1990
, “Mobile Robot Control, Part 1: Feedback Control of a Nonholo-Nomic Wheeled Cart in Cartesian Space
,” Institut National de Recherche en Informatique et en Automatique, Report 1288, Le Chesnay, France.31.
Aicardi
, M.
, Casalino
, G.
, Bicchi
, A.
, and Balestrino
, A.
, 1995
, “Closed Loop Steering of Unicycle Like Vehicles via Lyapunov Techniques
,” IEEE Rob. Autom. Mag.
, 2
(1
), pp. 27
–35
.10.1109/100.38829432.
Ren
, W.
, and Beard
, R.
, 2004
, “Trajectory Tracking for Unmanned Air Vehicles With Velocity and Heading Rate Constraints
,” IEEE Trans. Control Syst. Technol.
, 12
(5
), pp. 706
–716
.10.1109/TCST.2004.82695633.
Anderson
, R.
, Bakolas
, E.
, Milutinović
, D.
, and Tsio-tras
, P.
, 2013
, “Optimal Feedback Guidance of a Small Aerial Vehicle in a Stochastic Wind
,” J. Guidance Control Dyn.
, 36
(4
), pp. 975
–985
.10.2514/1.5951234.
Sutton
, R. S.
, and Barto
, A. G.
, 1998
, Reinforcement Learning: An Introduction
, Vol. 1
, The MIT Press, Cambridge, MA
.35.
Powell
, W. B.
, 2007
, Approximate Dynamic Programming: Solving the Curses of Dimensionality
, Vol. 703
, John Wiley & Sons
, Hoboken, NJ.10.1002/978111802917636.
Long
, A. W.
, Wolfe
, K. C.
, Mashner
, M. J.
, and Chirikjian
, G. S.
, 2012
, “The Banana Distribution is Gaussian: A Localization Study With Exponential Coordinates
,” Proceedings of Robotics: Science and Systems
, Syndey, Australia
, pp. 265–272.37.
Thrun
, S.
, Burgard
, W.
, and Fox
, D.
, 2005
, Probabilistic Robotics
, The MIT Press, Cambridge
, MA.38.
Anderson
, R. P.
, and Milutinović
, D.
, 2011
, “A Stochastic Approach to Dubins Feedback Control for Target Tracking
,” Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
, San Francisco, CA
, pp. 3917–3922.39.
Jadaliha
, M.
, Lee
, J.
, and Choi
, J.
, 2012
, “Adaptive Control of Multiagent Systems for Finding Peaks of Uncertain Static Fields
,” J. Dyn. Syst. Meas. Contr.
, 134
(5
), p. 051007
.10.1115/1.400636940.
Cressie
, N.
, 1986
, “Kriging Nonstationary Data
,” J. Am. Stat. Assoc.
, 81
(395
), pp. 625
–634
.10.1080/01621459.1986.1047831541.
Rasmussen
, C. E.
, and Williams
, C. K. I.
, 2006
, Gaussian Processes for Machine Learning
, The MIT Press, Cambridge
, MA
.42.
Choi
, J.
, Lee
, J.
, and Oh
, S.
, 2008
, “Biologically-Inspired Navigation Strategies for Swarm Intelligence Using Spatial Gaussian Processes
,” Proceedings of the 17th International Federation of Automatic Control (IFAC) World Congress
, Seoul, Korea.43.
Choi
, J.
, Lee
, J.
, and Oh
, S.
, 2008
, “Swarm Intelligence for Achieving the Global Maximum Using Spatio-Temporal Gaussian Processes
,” Proceedings of the 27th American Control Conference (ACC)
, Seattle, WA, pp. 135–140.44.
Shi
, J. Q.
, and Choi
, T.
, 2011
, Gaussian Process Regression Analysis for Functional Data
, CRC
, Boca Raton, FL.45.
Abrahamsen
, P.
, 1997
, A Review of Gaussian Random Fields and Correlation Functions
, Norsk Regnesentral/Norwegian Computing Center
, Oslo, Norway.46.
Snelson
, E. L.
, 2007
, “Flexible and Efficient Gaussian Process Models for Machine Learning
,” Ph.D. thesis, University College, London, UK.47.
Zhang
, B.
, and Sukhatme
, G.
, 2007
, “Adaptive Sampling for Estimating a Scalar Field Using a Robotic Boat and a Sensor Network
,” 2007 IEEE
International Conference on Robotics and Automation, Rome, Italy, Apr. 10–14, pp. 3673
–3680
.10.1109/ROBOT.2007.36404148.
Laut
, J.
, Henry
, E.
, Nov
, O.
, and Porfiri
, M.
, 2014
, “Development of a Mechatronics-Based Citizen Science Platform for Aquatic Environmental Monitoring
,” IEEE/ASME Trans. Mechatron.
, 19
(5
), pp. 1541
–1551
.10.1109/TMECH.2013.228770549.
Xu
, Y.
, and Choi
, J.
, 2011
, “Adaptive Sampling for Learning Gaussian Processes Using Mobile Sensor Networks
,” Sensors
, 11
(3
), pp. 3051
–3066
.10.3390/s11030305150.
Bishop
, C. M.
, 2006
, Pattern Recognition and Machine Learning
, Springer
, New York.
51.
Xu
, Y.
, Choi
, J.
, Dass
, S.
, and Maiti
, T.
, 2011
, “Bayesian Prediction and Adaptive Sampling Algorithms for Mobile Sensor Networks
,” Proceedings of the 2011 American Control Conference (ACC)
, San Francisco, CA, pp. 4095
–4200
.52.
Xu
, Y.
, 2011
, “Environmental Adaptive Sampling for Mobile Sensor networks using Gaussian processes
,” Ph.D. thesis, Michigan State University, East Lansing, MI.53.
Gaudard
, M.
, Karson
, M.
, Linder
, E.
, and Sinha
, D.
, 1999
, “Bayesian Spatial Prediction
,” Environ. Ecol. Stat.
, 6
(2
), pp. 147
–171
.10.1023/A:100961400369254.
Rue
, H.
, and Tjelmeland
, H.
, 2002
, “Fitting Gaussian Markov Random Fields to Gaussian Fields
,” Scand. J. Stat.
, 29
(1
), pp. 31
–49
.10.1111/1467-9469.0005855.
Cressie
, N.
, and Verzelen
, N.
, 2008
, “Conditional-Mean Least-Squares Fitting of Gaussian Markov Random Fields to Gaussian Fields
,” Comput. Stat. Data Anal.
, 52
(5
), pp. 2794
–2807
.10.1016/j.csda.2007.10.00856.
Hartman
, L.
, and Hössjer
, O.
, 2008
, “Fast Kriging of Large Data Sets With Gaussian Markov Random Fields
,” Comput. Stat. Data Anal.
, 52
(5
), pp. 2331
–2349
.10.1016/j.csda.2007.09.01857.
Le Ny
, J.
, and Pappas
, G. J.
, 2009
, “On Trajectory Optimization for Active Sensing in Gaussian Process Models
,” Proceedings of the 48th IEEE Conference on Decision and Control
, Shanghai, China, pp. 6286
–6292
.58.
Rasmussen
, C. E.
, and Nickisch
, H.
, 2010
, “Gaussian Processes for Machine Learning (GPML) Toolbox
,” J. Mach. Learn. Res.
, 11
, pp. 3011
–3015
.59.
Minka
, T. P.
, 2001
, “Expectation Propagation for Approximate Bayesian Inference
,”. Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence
, Morgan Kaufmann
, pp. 362
–369
.60.
Williams
, C. K.
, and Barber
, D.
, 1998
, “Bayesian Classification With Gaussian Processes
,” IEEE Trans. Pattern Anal. Mach. Intell.
, 20
(12
), pp. 1342
–1351
.10.1109/34.735807Copyright © 2015 by ASME
You do not currently have access to this content.
