Abstract
Technology advancement for on-road vehicles has gained significant momentum in the past decades, particularly in the field of vehicle automation and powertrain electrification. The optimization of powertrain controls for autonomous vehicles typically involves a separated consideration of the vehicle’s external dynamics and powertrain dynamics, with one key aspect often overlooked. This aspect, known as flexible power demand, recognizes that the powertrain control system does not necessarily have to precisely match the power requested by the vehicle motion controller at all times. Leveraging this feature can lead to control designs achieving improved fuel economy by adding an extra degrees-of-freedom to the powertrain control while maintaining safety and drive comfort. The present research investigates the use of an approximate dynamic programming (ADP) approach to develop a powertrain controller, which takes into account the flexibility in power demand within the ADP framework. The concept of reachable sets is incorporated into the ADP framework to ensure safety, improve ride comfort, and enhance the accuracy of the optimization solution. The formulation is based on an autonomous hybrid electric vehicle, while the methodology can also be applied to other types of vehicles. It is also found that necessary customization of the ADP algorithm is needed for this particular control problem to prevent convergence issues. Finally, a case study is presented to evaluate the effectiveness of flexible power demand, as addressed by the ADP method. The experiment demonstrates a 14.1% improvement in fuel economy compared to a scenario without flexible power demand.
References
1.
Kargar
, M.
, and Song
, X.
, 2022
, “Power Control Optimization for Autonomous Hybrid Electric Vehicles With Flexible Driveline Torque Demand
,” 2022 American Control Conference (ACC)
, Atlanta, GA
, IEEE
, pp. 2012
–2017
.2.
Qu
, X.
, Yu
, Y.
, Zhou
, M.
, Lin
, C.-T.
, and Wang
, X.
, 2020
, “Jointly Dampening Traffic Oscillations and Improving Energy Consumption With Electric, Connected and Automated Vehicles: A Reinforcement Learning Based Approach
,” Appl. Energy
, 257
, p. 114030
. 3.
Zhang
, F.
, Wang
, L.
, Coskun
, S.
, Pang
, H.
, Cui
, Y.
, and Xi
, J.
, 2020
, “Energy Management Strategies for Hybrid Electric Vehicles: Review, Classification, Comparison, and Outlook
,” Energies
, 13
(13
), p. 3352
. 4.
Kavas-Torris
, O.
, Cantas
, M. R.
, Meneses Cime
, K.
, Aksun Guvenc
, B.
, and Guvenc
, L.
, 2020
, The Effects of Varying Penetration Rates of l4-l5 Autonomous Vehicles on Fuel Efficiency and Mobility of Traffic Networks, Technical Report.5.
Duarte
, F.
, and Ratti
, C.
, 2018
, “The Impact of Autonomous Vehicles on Cities: A Review
,” J. Urban Technol.
, 25
(4
), pp. 3
–18
. 6.
Doshi
, P.
, Kapur
, D.
, and Iyer
, R.
, 2017
, “Hybridization-Bridge for Electrification
,” 2017 IEEE Transportation Electrification Conference (ITEC—India)
, Pune, India
, IEEE
, pp. 1
–5
.7.
Liu
, J.
, and Peng
, H.
, 2008
, “Modeling and Control of a Power-Split Hybrid Vehicle
,” IEEE Trans. Control Syst. Technol.
, 16
(6
), pp. 1242
–1251
. 8.
Hong
, S.
, Kim
, H.
, and Kim
, J.
, 2015
, “Motor Control Algorithm for an Optimal Engine Operation of Power Split Hybrid Electric Vehicle
,” Int. J. Autom. Technol.
, 16
(1
), pp. 97
–105
. 9.
Borhan
, H. A.
, Vahidi
, A.
, Phillips
, A. M.
, Kuang
, M. L.
, and Kolmanovsky
, I. V.
, 2009
, “Predictive Energy Management of a Power-Split Hybrid Electric Vehicle
,” 2009 American Control Conference
, St. Louis, MO
, IEEE
, pp. 3970
–3976
.10.
Wang
, R.
, and Lukic
, S. M.
, 2012
, “Dynamic Programming Technique in Hybrid Electric Vehicle Optimization
,” 2012 IEEE International Electric Vehicle Conference
, Greenville, SC
, IEEE
, pp. 1
–8
.11.
Pérez
, L. V.
, Bossio
, G. R.
, Moitre
, D.
, and García
, G. O.
, 2006
, “Optimization of Power Management in an Hybrid Electric Vehicle Using Dynamic Programming
,” Math. Comput. Simul.
, 73
(1–4
), pp. 244
–254
. 12.
Zeng
, X.
, and Wang
, J.
, 2015
, “A Parallel Hybrid Electric Vehicle Energy Management Strategy Using Stochastic Model Predictive Control With Road Grade Preview
,” IEEE Trans. Control Syst. Technol.
, 23
(6
), pp. 2416
–2423
. 13.
Huang
, Y.
, Wang
, H.
, Khajepour
, A.
, He
, H.
, and Ji
, J.
, 2017
, “Model Predictive Control Power Management Strategies for HEVs: A Review
,” J. Power Sources
, 341
, pp. 91
–106
. 14.
Hofman
, T.
, Steinbuch
, M.
, Van Druten
, R.
, and Serrarens
, A.
, 2007
, “Rule-Based Energy Management Strategies for Hybrid Vehicles
,” Int. J. Electr. Hybrid Vehicles
, 1
(1
), pp. 71
–94
. 15.
Jalil
, N.
, Kheir
, N. A.
, and Salman
, M.
, 1997
, “A Rule-Based Energy Management Strategy for a Series Hybrid Vehicle
,” Proceedings of the 1997 American Control Conference (Cat. No. 97CH36041)
, Albuquerque, NM
, Vol. 1
, IEEE
, pp. 689
–693
.16.
Paganelli
, G.
, Delprat
, S.
, Guerra
, T.-M.
, Rimaux
, J.
, and Santin
, J.-J.
, 2002
, “Equivalent Consumption Minimization Strategy for Parallel Hybrid Powertrains
,” Vehicular Technology Conference. IEEE 55th Vehicular Technology Conference. VTC Spring 2002 (Cat. No. 02CH37367)
, Birmingham, AL
, Vol. 4
, IEEE
, pp. 2076
–2081
.17.
Škugor
, B.
, Deur
, J.
, Cipek
, M.
, and Pavković
, D.
, 2014
, “Design of a Power-Split Hybrid Electric Vehicle Control System Utilizing a Rule-Based Controller and an Equivalent Consumption Minimization Strategy
,” Proc. Inst. Mech. Eng., Part D: J. Automobile Eng.
, 228
(6
), pp. 631
–648
. 18.
Yuan
, Z.
, Teng
, L.
, Fengchun
, S.
, and Peng
, H.
, 2013
, “Comparative Study of Dynamic Programming and Pontryagin’s Minimum Principle on Energy Management for a Parallel Hybrid Electric Vehicle
,” Energies
, 6
(4
), pp. 2305
–2318
. 19.
Jeong
, J.
, Lee
, D.
, Kim
, N.
, Zheng
, C.
, Park
, Y.-I.
, and Cha
, S. W.
, 2014
, “Development of PMP-Based Power Management Strategy for a Parallel Hybrid Electric Bus
,” Int. J. Precis. Eng. Manuf.
, 15
(2
), pp. 345
–353
. 20.
Ahmadizadeh
, P.
, Mashadi
, B.
, and Lodaya
, D.
, 2017
, “Energy Management of a Dual-Mode Power-Split Powertrain Based on the Pontryagin’s Minimum Principle
,” IET Intell. Transp. Syst.
, 11
(9
), pp. 561
–571
. 21.
Lian
, R.
, Peng
, J.
, Wu
, Y.
, Tan
, H.
, and Zhang
, H.
, 2020
, “Rule-Interposing Deep Reinforcement Learning Based Energy Management Strategy for Power-Split Hybrid Electric Vehicle
,” Energy
, 197
, p. 117297
. 22.
Wu
, J.
, He
, H.
, Peng
, J.
, Li
, Y.
, and Li
, Z.
, 2018
, “Continuous Reinforcement Learning of Energy Management With Deep Q Network for a Power Split Hybrid Electric Bus
,” Appl. Energy
, 222
, pp. 799
–811
. 23.
Yazar
, O.
, Coskun
, S.
, Li
, L.
, Zhang
, F.
, and Huang
, C.
, 2023
, “Actor-Critic TD3-Based Deep Reinforcement Learning for Energy Management Strategy of HEV
,” 2023 5th International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA)
, Istanbul, Turkey
, IEEE
, pp. 1
–6
.24.
Fagnant
, D. J.
, and Kockelman
, K.
, 2015
, “Preparing a Nation for Autonomous Vehicles: Opportunities, Barriers and Policy Recommendations
,” Transp. Res. Part A: Policy Practice
, 77
, pp. 167
–181
. 25.
Rashid
, M. M.
, Farzaneh
, F.
, Seyedi
, M.
, and Jung
, S.
, 2023
, “Evaluation of Risk Injury in Pedestrians’ Head and Chest Region During Collision With an Autonomous Bus
,” Int. J. Crashworthiness
, 29
(2
), pp. 1
–11
. 26.
Martínez-Díaz
, M.
, and Soriguera
, F.
, 2018
, “Autonomous Vehicles: Theoretical and Practical Challenges
,” Transp. Res. Procedia
, 33
, pp. 275
–282
. 27.
Litman
, T.
, 2020
, “Autonomous Vehicle Implementation Predictions: Implications for Transport Planning”.28.
Ye
, L.
, and Yamamoto
, T.
, 2019
, “Evaluating the Impact of Connected-Dissipation of Stop-and-Go Waves Via and Autonomous Vehicles on Traffic Safety
,” Physica A: Stat. Mech. Appl.
, 526
, p. 121009
. 29.
Stern
, R. E.
, Cui
, S.
, Delle Monache
, M. L.
, Bhadani
, R.
, Bunting
, M.
, Churchill
, M.
, Hamilton
, N.
, et al., 2018
, “Dissipation of Stop-and-Go Waves Via Control of Autonomous Vehicles: Field Experiments
,” Transp. Res. Part C: Emerg. Technol.
, 89
, pp. 205
–221
. 30.
Ghasemi
, M.
, and Song
, X.
, 2018
, “Powertrain Energy Management for Autonomous Hybrid Electric Vehicles With Flexible Driveline Power Demand
,” IEEE Trans. Control Syst. Technol.
, 27
(5
), pp. 2229
–2236
. 31.
Panday
, A.
, and Bansal
, H. O.
, 2014
, “A Review of Optimal Energy Management Strategies for Hybrid Electric Vehicle
,” Int. J. Vehicular Technol.
, 2014
(1
), p. 160510
. 32.
Zulkefli
, M. A. M.
, Zheng
, J.
, Sun
, Z.
, and Liu
, H. X.
, 2014
, “Hybrid Powertrain Optimization With Trajectory Prediction Based on Inter-Vehicle-Communication and Vehicle-Infrastructure-Integration
,” Transp. Res. Part C: Emerg. Technol.
, 45
, pp. 41
–63
. 33.
Kim
, N.
, Cha
, S. W.
, and Peng
, H.
, 2011
, “Optimal Equivalent Fuel Consumption for Hybrid Electric Vehicles
,” IEEE Trans. Control Syst. Technol.
, 20
(3
), pp. 817
–825
. 34.
Kim
, H.
, and Kum
, D.
, 2016
, “Comprehensive Design Methodology of Input-and Output-Split Hybrid Electric Vehicles: In Search of Optimal Configuration
,” IEEE/ASME Trans. Mechatron.
, 21
(6
), pp. 2912
–2923
. 35.
Kim
, S. J.
, Kim
, K. -S.
, and Kum
, D.
, 2015
, “Feasibility Assessment and Design Optimization of a Clutchless Multimode Parallel Hybrid Electric Powertrain
,” IEEE/ASME Trans. Mechatron.
, 21
(2
), pp. 774
–786
. 36.
Kim
, N.
, Rousseau
, A.
, and Lee
, D.
, 2011
, “A Jump Condition of PMP-Based Control for PHEVs
,” J. Power Sources
, 196
(23
), pp. 10380
–10386
. 37.
Anderson
, S. J.
, Peters
, S. C.
, Pilutti
, T. E.
, and Iagnemma
, K.
, 2010
, “An Optimal-Control-Based Framework for Trajectory Planning, Threat Assessment, and Semi-Autonomous Control of Passenger Vehicles in Hazard Avoidance Scenarios
,” Int. J. Vehicle Autonom. Syst.
, 8
(2–4
), pp. 190
–216
. 38.
Foderaro
, G.
, Ferrari
, S.
, and Wettergren
, T. A.
, 2014
, “Distributed Optimal Control for Multi-agent Trajectory Optimization
,” Automatica
, 50
(1
), pp. 149
–154
. 39.
Jantapremjit
, P.
, and Wilson
, P. A.
, 2007
, “Control and Guidance for Homing and Docking Tasks Using an Autonomous Underwater Vehicle
,” 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems
, San Diego, CA
, IEEE
, pp. 3672
–3677
.40.
Ma
, J.
, Zheng
, Y.
, and Wang
, L.
, 2015
, “LQR-Based Optimal Topology of Leader-Following Consensus
,” Int. J. Robust Nonlinear Control
, 25
(17
), pp. 3404
–3421
. 41.
Zhang
, H.
, Feng
, T.
, Yang
, G.-H.
, and Liang
, H.
, 2014
, “Distributed Cooperative Optimal Control for Multiagent Systems on Directed Graphs: An Inverse Optimal Approach
,” IEEE Trans. Cybern.
, 45
(7
), pp. 1315
–1326
. 42.
Zhang
, H.
, Zhang
, J.
, Yang
, G.-H.
, and Luo
, Y.
, 2014
, “Leader-Based Optimal Coordination Control for the Consensus Problem of Multiagent Differential Games Via Fuzzy Adaptive Dynamic Programming
,” IEEE Trans. Fuzzy Syst.
, 23
(1
), pp. 152
–163
. 43.
Yao
, Q.
, Tian
, Y.
, Wang
, Q.
, and Wang
, S.
, 2020
, “Control Strategies on Path Tracking for Autonomous Vehicle: State of the Art and Future Challenges
,” IEEE Access
, 8
, pp.161211
–161222
. 44.
Atkinson
, C.
, Lewis
, A.
, Salvia
, A.
, and Vishwanathan
, G.
, 2015
, “Powertrain Innovations for Connected and Autonomous Vehicles
,” Proceedings of Powertrain Innovations Workshop, Advanced Research Projects Agency -Energy, pp. 1
–8
.45.
Kargar
, M.
, Zhang
, C.
, and Song
, X.
, 2022
, “Integrated Optimization of Powertrain Energy Management and Vehicle Motion Control for Autonomous Hybrid Electric Vehicles
,” 2022 American Control Conference (ACC)
, Atlanta, GA
, pp. 404
–409
.46.
Zhang
, L.
, Ye
, X.
, Xia
, X.
, and Barzegar
, F.
, 2020
, “A Real-Time Energy Management and Speed Controller for an Electric Vehicle Powered by a Hybrid Energy Storage System
,” IEEE Trans. Ind. Inf.
, 16
(10
), pp. 6272
–6280
. 47.
Wang
, W.
, Guo
, X.
, Yang
, C.
, Zhang
, Y.
, Zhao
, Y.
, Huang
, D.
, and Xiang
, C.
, 2022
, “A Multi-objective Optimization Energy Management Strategy for Power Split HEV Based on Velocity Prediction
,” Energy
, 238
, p. 121714
. 48.
Zhao
, L.
, Mahbub
, A. I.
, and Malikopoulos
, A. A.
, 2019
, “Optimal Vehicle Dynamics and Powertrain Control for Connected and Automated Vehicles
,” 2019 IEEE Conference on Control Technology and Applications (CCTA)
, Hong Kong, China
, IEEE
, pp. 33
–38
.49.
Mahbub
, A.
, and Malikopoulos
, A. A.
, 2019
, “Concurrent Optimization of Vehicle Dynamics and Powertrain Operation Using Connectivity and Automation”. arXiv preprint arXiv:1911.03475.50.
Ma
, G.
, Ghasemi
, M.
, and Song
, X.
, 2017
, “Integrated Powertrain Energy Management and Vehicle Coordination for Multiple Connected Hybrid Electric Vehicles
,” IEEE Trans. Vehicular Technol.
, 67
(4
), pp. 2893
–2899
. 51.
Engbroks
, L.
, Görke
, D.
, Schmiedler
, S.
, Strenkert
, J.
, and Geringer
, B.
, 2018
, “Applying Forward Dynamic Programming to Combined Energy and Thermal Management Optimization of Hybrid Electric Vehicles
,” IFAC-PapersOnLine
, 51
(31
), pp. 383
–389
. 52.
Zhang
, F.
, Hu
, X.
, Langari
, R.
, Wang
, L.
, Cui
, Y.
, and Pang
, H.
, 2021
, “Adaptive Energy Management in Automated Hybrid Electric Vehicles With Flexible Torque Request
,” Energy
, 214
, p. 118873
. 53.
Sánchez
, M.
, Delprat
, S.
, and Hofman
, T.
, 2020
, “Energy Management of Hybrid Vehicles With State Constraints: A Penalty and Implicit Hamiltonian Minimization Approach
,” Appl. Energy
, 260
, p. 114149
. 54.
Serrao
, L.
, Onori
, S.
, and Rizzoni
, G.
, 2009
, “ECMS As a Realization of Pontryagin’s Minimum Principle for HEV Control
,” In 2009 American control conference, IEEE
, pp. 3964
–3969
.55.
Elbert
, P.
, Ebbesen
, S.
, and Guzzella
, L.
, 2012
, “Implementation of Dynamic Programming for -Dimensional Optimal Control Problems With Final State Constraints
,” IEEE Trans. Control Syst. Technol.
, 21
(3
), pp. 924
–931
. 56.
Kargar
, M.
, Zhang
, C.
, and Song
, X.
, 2023
, “Integrated Optimization of Power Management and Vehicle Motion Control for Autonomous Hybrid Electric Vehicles
,” IEEE Trans. Vehicular Technol.
, 72
(9
), pp. 11147
–11155
. 57.
Kim
, N.
, Cha
, S.
, and Peng
, H.
, 2010
, “Optimal Control of Hybrid Electric Vehicles Based on Pontryagin’s Minimum Principle
,” IEEE Trans. Control Syst. Technol.
, 19
(5
), pp. 1279
–1287
. 58.
Bellman
, R.
, 1966
, “Dynamic Programming
,” Science
, 153
(3731
), pp. 34
–37
. 59.
Lewis
, F. L.
, and Vrabie
, D.
, 2009
, “Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control
,” IEEE Circu. Syst. Mag.
, 9
(3
), pp. 32
–50
. 60.
Heydari
, A.
, and Balakrishnan
, S. N.
, 2012
, “Finite-Horizon Control-Constrained Nonlinear Optimal Control Using Single Network Adaptive Critics
,” IEEE Trans. Neural Netw. Learn. Syst.
, 24
(1
), pp. 145
–157
. 61.
Li
, C.
, Ding
, J.
, Lewis
, F. L.
, and Chai
, T.
, 2021
, “A Novel Adaptive Dynamic Programming Based on Tracking Error for Nonlinear Discrete-Time Systems
,” Automatica
, 129
, p. 109687
. 62.
Kiumarsi
, B.
, AlQaudi
, B.
, Modares
, H.
, Lewis
, F. L.
, and Levine
, D. S.
, 2019
, “Optimal Control Using Adaptive Resonance Theory and Q-Learning
,” Neurocomputing
, 361
, pp. 119
–125
. 63.
Abu-Khalaf
, M.
, and Lewis
, F. L.
, 2005
, “Nearly Optimal Control Laws for Nonlinear Systems With Saturating Actuators Using a Neural Network HJB Approach
,” Automatica
, 41
(5
), pp. 779
–791
. 64.
Khajeh-Hosseini
, M.
, and Talebpour
, A.
, 2021
, “A Novel Clustering Approach to Identify Vehicles Equipped With Adaptive Cruise Control in a Vehicle Trajectory Data
,” 100th Annual Meeting of the Transportation Research Board of National Academies, Washington, DC.65.
Kargar
, M.
, Sardarmehni
, T.
, and Song
, X.
, 2022
, “Optimal Powertrain Energy Management for Autonomous Hybrid Electric Vehicles With Flexible Driveline Power Demand Using Approximate Dynamic Programming
,” IEEE Trans. Vehicular Technol.
, 71
(12
), pp. 12564
–12575
. 66.
Onori
, S.
, Serrao
, L.
, and Rizzoni
, G.
, 2016
, “Hybrid Electric Vehicles: Energy Management Strategies”.Copyright © 2024 by ASME
You do not currently have access to this content.
