## Abstract

Combined powertrain and velocity optimization can achieve significant energy efficiency improvements. However, due to the multitime scales in the system, the optimization is performed hierarchically and by separating time scales. To enforce state constraints, iteration between controller is introduced, for example, using Lagrange multipliers as metric for constraint violation. In this paper, an extension of the Koopman operator theory is presented with to obtain a data-driven approximation of the multipliers' behavior hence eliminating the need for iterations. Because the evolution of the Lagrange multipliers is the result of a fast dynamics optimization problem, and not the response of a nonlinear dynamical system, a novel technique in which the Lagrange multipliers are interpreted as a dynamic system is presented here. The approximate Koopman linear system is then derived using extended dynamic mode decomposition and it is integrated with the slow dynamic optimization. Results show that the Koopman augmented controller, which is solved as one single optimization, meets state and input constraints and achieves similar energy savings compared to an iterative approach.

## 1 Introduction

Class-8 trucks are responsible for approximately 20% of the total highway fuel consumption [1]. Hence, improvement in energy efficiency of commercial vehicle is crucial for addressing the sustainability challenges of the transportation sector. The repetitive nature of trucking routes lends itself to automation, potentially allowing semitrucks to take advantage of the recent developments in connected and autonomous vehicle (CAV) technology. Specifically, control strategies have been developed that utilize GPS, traffic, and road topography information available to CAVs to save energy, including vehicle speed optimization [2–5], eco-routing [6], platooning [7,8], gear selection optimization [9,10], optimal traffic signal arrival and departure [11,12], and engine and cabin thermal management [13,14]. When integrated, these strategies have been shown to lead to an average of 20% energy usage reduction on a variety of vehicle platforms [4,5].

One of the challenges of developing strategies that combine both, velocity optimization and powertrain control, is that the plant model is characterized by multiple time-scales. Singularly perturbed problems can be solved efficiently by partitioning the plant based on time-scale and solving the coupled subproblems independently, where, for a two-time-scales systems, the slower dynamics handled by the higher layer controller, and the faster dynamics handled by the lower layer controller. Hierarchical controllers can be categorized by the level of communication between layers, either no communication, one-way communication, and two-way communication. When coupling between the two time-scales is weak, a hierarchical controller with no communication between the layers can be used [15,16]. In most scenarios, coupling between time-scales is stronger, and requires some communication between the controllers. Most hierarchical controllers pass information in one direction from the higher to the lower layer where the higher layer controller optimizes a cost function over a longer horizon, while neglecting the fast dynamics, such as actuator dynamics. The output from the higher layer controller acts as a reference that the lower layer controller attempts to track. Other applications solve an optimization problem in both layers where the computation time is reduced by, for example, using different temporal discretizations in each layers [17] or using different fidelity models in each layers [18].

A challenge arises when the lower layer is unable to track the profile computed by the higher layer, which can often occur, for example, in heavy duty powertrain control where the engine is not capable to provide enough change in torque to meet the required acceleration. One approach consists in tightening the constraints on the higher layer such that the lower layer tracking problem remains feasible [19]. Other approaches that additional communication and iteration are based on treating the deviation of the lower layer from the desired setpoint as a disturbance to be rejected [20], on recomputing the solution of the higher level if the tracking is unfeasible [21], or on allowing deviation from the reference provided that some metrics on the slow-dynamics problem remain unchanged [22]. None of these approaches are systematic in the way the infeasibility is addressed, which will lead to a large optimality gap. Few examples exist where tracking failure is addressed by feedback from the lower layer. In Ref. [23], the higher layer provides a reference for the lower layer, and a constraint on the deviation from the reference. If the lower layer is unable to meet the relaxed tracking constraint, a binary variable is fed back to the higher layer, which is used to lower the performance and to trigger a recomputation of the profile. A similar approach is presented in [24], where if the lower layer violates this constraint, the penalty terms in the higher layer objective function are modified to favor a smoother profile, rather than an economic one. While simulation results are promising, the weight tuning process is entirely empirical and requires extensive calibration. In this context, a novel iterative hierarchical scheme for multitime scale systems has been presented by the authors in Ref. [25]. The strategy has the ability to guarantee constraint satisfaction across time-scales, while obtaining a near-optimal solution comparable to the centralized solution at a significantly reduced computation time. This is achieved by using the Lagrange multipliers of the fast dynamic as measure for constraints satisfaction in the slow dynamic optimization. Specifically, by using the properties of Lagrange multipliers that are nonzero if and only if the corresponding constraints are active, and their magnitude is a measure of the distance to the constraint. The main limitation of this approach is the need for an iterative process with unknown convergence rate, which in general will result in increased computational burden. This paper presents an extension of the iterative model predictive control (MPC) scheme presented in Ref. [25] based on the hypothesis that the fast dynamics Lagrange multipliers are described by an unknown nonlinear dynamic system that can be identified using Koopman operator theory. The resulting linear system is then integrated in the higher layer controller to predict the fast dynamics Lagrange multipliers without iteration. A simulation study is shown in this paper for the combined velocity and powertrain optimization in a heavy duty vehicle. The simulation results compare the iterative solution against the sequential method that uses the Koopman system. Results show that the Koopman-augmented MPC achieves computationally efficient near-optimal results. This paper is structured as follows. First, the optimal control problem is defined for the case of a general two-time scales system with coupled state constraints. Then an overview of Koopman theory is introduced and its application to the identification of the Lagrange is provided. Finally, the case study of the connected vehicle is introduced and the new method is compared against a traditional iterative approach.

## 2 Control Problem Definition

### 2.1 Summary of the Plant Model Equations.

The vehicle considered in this paper is a Volvo VNL 300 class 8 semitruck with a 13 liter six cylinder turbocharged Diesel engine and a twelve speed automated manual transmission.

*i*is indicates either air or fuel. The time constants

*τ*are function of the change in torque request $\Delta Tc$ and are calibrated to meet the transient responses collected on the engine dynamometer and lump the effect of air and fuel path components. The steady-state fuel flowrate $m\u02d9f,ss$ is mapped as function of desired engine torque and engine speed, while the steady-state air mass flowrate is given by the speed density equation

_{i}*V*is engine displacement,

_{d}*N*is the engine speed,

*R*is the specific gas constant for air,

*p*is the intake manifold pressure,

*T*

_{im}is the intake manifold temperature, which are mapped based on engine speed and torque request, and

*λ*is volumetric efficiency [27]

_{v}*k*

_{1}and

*k*

_{2}are function of engine speed [25]. In Diesel Engines, to prevent smoke and limit emission a saturation on the fuel flowrate that is implemented

_{lim}represents the maximum FAR and, in this paper, a constant value of $5.9\xd710\u22122$ has been used [25,28]. The engine torque is calculated as

*F*is calculated assuming that all shafts are rigid, the gear efficiency is constant and gear changes are instantaneous and the wheel speed is an algebraic function of the vehicle speed

_{t}*v*. Finally, the road load equation is used to determine vehicle speed

where *m*_{eff} is the vehicle effective mass, *g* is the gravitational acceleration, *θ* is the road grade, *c _{r}* is the rolling resistance coefficient,

*ρ*is the density of air,

*C*is the drag coefficient, and

_{d}*A*is the frontal area. A complete validation of the model against engine dynamometer and road data is shown in Ref. [25].

_{f}*x*and control input

*u*are defined as

where *d* is the distance, *v* is the vehicle speed, $m\u02d9f$ and $m\u02d9air$ are the fuel and air flowrate, respectively, *T _{c}* is the engine torque request, $\Delta Tc$ is the change in the control command and the subscript

*k*is the time step. For the control design, the system is augmented to account for traveled distance and rate of change in the control input for drivability [25,29,30].

### 2.2 Centralized Problem.

*d*

_{ref}over a given time imposed by a supervisory controller. The optimal control sequence is obtained by solving the following finite horizon optimization problem

where *N _{p}* is the length of the horizon, $\Delta t$ is the discretization time,

*γ*

_{1},

*γ*

_{2}, and

*γ*

_{3}are weights penalizing fuel flowrate, change in torque command, and the distance traveled at the final time, respectively. The weights are selected though a calibration process to achieve a desired tradeoff, for example, optimal fuel economy with no changes in travel time [5]. The function

*g*includes the state and input constraints, such as fuel-to-air constraint that limits the change in wheel torque due to availability of fuel and air as defined in Eq. (4). The dynamics in the centralized problems have two different time-scales. Namely, a slow dynamics associated with the vehicle longitudinal dynamic, and a fast dynamics associated with the engine transients.

### 2.3 Iterative Solution Via Communication of the Lagrange Multipliers.

*s*and

*f*refer to the slow and fast dynamics, respectively. The states and control inputs for each time-scales are

*N*is the horizon for the slow dynamics optimization. The function

_{s}*g*includes the state and input constraints and $\Delta ts$ is the discretization time used for the slow dynamic. Finally, $x\xaff$ is defined as

^{s}*N*is the length of horizon for the fast dynamic problem and $x\xafs$ are the nominal trajectories of the slow states obtained from the slow dynamic optimization problem and are included as external inputs. The function

_{f}*g*includes the state, input, and the FAR constraints. While it is clear that solving the problem sequentially will improve computation time and possibly lead to a real-time executable algorithm [23], this approach cannot guarantee that the optimal trajectories for $m\u02d9f,kref$ are feasible due to the constraints on the fast dynamics [29]. Specific to the heavy duty control problem, an increased torque demand is met by increasing the amount of fuel to the engine. However, as shown in Eq. (4), this is limited by the amount of fresh air that can be induced in the cylinder and air and fuel are characterized by a different dynamic response leading to constraint violation. To address this issue, a novel iterative approach based on the Lagrange multipliers was presented in Ref. [25] and verified on selected maneuvers that require rapid changes in torque production. The optimal Lagrange multipliers are defined in vector form as

^{f}where $u*$ is the optimal control sequence, *G* is the function of the constraints, and *J* is the objective function. The Lagrange multipliers *λ* represent the change of the objective function due to a perturbation of the constraints [32,33]. This concept is leveraged in the iterative method developed in Ref. [25] to find the minimum value of $\gamma 2s$, such that $\u2211\lambda *$ is as small as possible. This corresponds to a solution of the slow time-scale optimization problem that is feasible for the fast dynamics, and associated with the smallest penalty on the actuation.

which include state, input, and FAR constraints. The iterative approximation method for estimating the behavior of the optimal Lagrange multiplier $\lambda *$ as function of *R _{s}* presented is summarized in Algorithm 1 [25].

1: q = 0 |

2: while$\u2211\lambda >\lambda max\u2009\u2009&q<qmax$do |

3: ifq < 2 then |

4: R is user defined_{s} |

5: else if$q<q\u2009exp\u2009$then |

6: Choose R s.t. $\u2211\lambda \xafl(Rs)=0$, using a linear fit for $\u2211\lambda \xafl$_{s} |

7: else |

8: Choose R s.t. $\u2211\lambda \xafe(Rs)\u2264\lambda max$, using an exponential fit for $\u2211\lambda \xafe$_{s} |

9: end if |

10: $q++$ |

11: end while |

1: q = 0 |

2: while$\u2211\lambda >\lambda max\u2009\u2009&q<qmax$do |

3: ifq < 2 then |

4: R is user defined_{s} |

5: else if$q<q\u2009exp\u2009$then |

6: Choose R s.t. $\u2211\lambda \xafl(Rs)=0$, using a linear fit for $\u2211\lambda \xafl$_{s} |

7: else |

8: Choose R s.t. $\u2211\lambda \xafe(Rs)\u2264\lambda max$, using an exponential fit for $\u2211\lambda \xafe$_{s} |

9: end if |

10: $q++$ |

11: end while |

While this approach has been shown to satisfy inequality constraints in multitime-scale optimal control problems without penalizing system performance with near real-time performance, the iteration required to approximate the Lagrange multipliers increases computation requirements compared to a sequential scheme [25]. One possible approach to eliminate the need for iteration is to perform a design of experiment (DOE), where all the possible combinations of output reference and operating conditions are explored and a static map for $\gamma 2s$ is stored. While conceptually simple, this calibration-based approach relies on a precomputed map, which might not perform well for combination of points that were not part of the original DOE. In addition, the evaluation of the DOE requires an extensive simulation campaign. An alternative approach to the mapping of $\gamma 2s$ consists in exploring data driven methods. In this paper, the Koopman operator theory is extended to the problem of predicting Lagrange multipliers.

## 3 Overview of Koopman Operator Theory and Dynamic Mode Decomposition

Finding a functional approximation for $\lambda *$ will allow to combine the benefit of an iterative scheme with the computational efficiency of a sequential approach. The simplest possible data-driven approach consists of training of a neural network [34,35]. However, using a neural network as the prediction function introduces nonlinearity and nonconvexity to the slow dynamics optimization problem, requiring more complicated and time-consuming algorithms to solve, such as dynamic programing [36]. Koopman operator theory offers a promising pathways for real-time implementation as it consists of a methodology to describe nonlinear dynamics using a higher dimensional linear model [37–41]. While this approach will increase the dimension of the state vector in the slow dynamic optimization problem, it retains the model convexity and hence allows for adopting computationally efficient quadratic programing solvers.

### 3.1 Koopman Operator Background.

*g*are also known as observable functions. The Koopman operator $K$ is an infinite dimensional linear operator that acts on the measurement function

*g*such that

*λ*is the corresponding eigenvalue. The eigenfunctions can be used to construct the set of observable functions

*g*, given by

*v*is the Koopman mode given by

_{j}where *A* and *B* are the infinite dimensional Koopman linear system matrices and *u* is the original control input.

Whether the system is autonomous or not, in many scenarios no Koopman invariant subspace exists and an exact finite dimensional Koopman linear system cannot be found. Furthermore, for the application of Lagrange multiplier prediction, the governing nonlinear equations are unknown, making it impossible to postulate an analytical expression for the eigenfunctions. In this scenario, a data driven approach is the only viable solution to determine the Koopman eigenfunctions.

### 3.2 Data Driven Approximation of the Koopman Operator.

*n*+

*1 is the total number of samples of*

*x*collected. The states in

*X*and inputs

*U*at each index represent the conditions at some instant

*k*. The Koopman operator is calibrated such that the discrete Koopman linear system predicts the value in

*X*

^{+}at the same index, corresponding to the time instant

*k*+

*1. The candidate observable functions are then applied to these snapshot states to find the snapshot observable functions, given by*

where $||\xb7||F$ is the Frobenius norm.

## 4 Application to the Connected and Autonomous Vehicle Problem

While the popularity of Koopman operator theory for control has grown recently, its application has been restricted to identify linear models of nonlinear dynamical systems [38–44,46,47]. However, for the CAV problem presented in this paper and the issue associated with the identification of the Lagrange multipliers, a further extension of the method is needed. In this case, the evolution of the Lagrange multipliers is a result of the fast dynamics optimization problem, and not the response of a nonlinear dynamical system. Therefore, an novel technique in which the Lagrange multipliers are interpreted as a dynamic system is presented in this paper, and an approximate Koopman linear system is derived using EDMD.

### 4.1 Lagrange Multipliers as a Dynamic System.

*k*. Therefore, there is an associated Lagrange multiplier at each fast dynamic time-step. In order to represent the Lagrange multipliers at the slow dynamics time-scale, the value $\lambda \u0302k$ is defined as the sum of Langrange multipliers over one slow dynamics time-step. This concept is illustrated in Fig. 1. The red circles indicate the Lagrange multipliers sampled at the original fast dynamics time-step, while the green diamonds indicate the sum of the Lagrange multipliers over the interval between two slow dynamics sampling times. This procedure can also be expressed as

where *k* is the time index of the slow dynamic system, *j* is the time index of the fast dynamic system, and *κ* is the ratio of the slow time-step size $\Delta ts$ to the fast time-step size $\Delta tf$.

and it is assumed the function $f\lambda \u0302$ exists, it is both unknown and nonlinear. Hence, an exact finite set of eigenfunction cannot be determined analytically and a data driven approach is required.

### 4.2 Koopman Operator Model Fitting Using EDMD.

where *ϵ* is a calibrated shaping parameter and $\lambda \u0302ci$ are the radial centers of the data clusters. The values of the hyperparameters *p _{m}* and

*p*are calibrated through an exhaustive search, where the problem in Eq. (45) is solved using a subset of the snapshot matrices for various combinations of

_{g}*p*and

_{m}*p*. The remaining portion of the snapshot is used to validate the accuracy of the Koopman linear system compared to the original system. Following the calibration procedure, the values of

_{g}*p*and

_{m}*p*are 2 and 9, respectively. For the current set of observable functions, the calibration procedure required about one day on a desktop PC running an Intel

_{g}^{®}Xeon

^{®}CPU E5-1650 v2 at 3.5 GHz with 64 GB. However, this time can be reduced by using higher power computational resources, i.e., a supercomputer, or by evaluating each set of hyperparameters in parallel.

The comparison of the Koopman predicted value of the Lagrange multipliers against the true output is shown in Fig. 2. While the error is largely within $\xb110%$ and the prediction has an root mean square (RMS) of 10.9, there is an significant number of points that fall outside this range. This indicates that the chosen candidate observable functions are insufficient, and an expansion of function dictionary is required for improving the accuracy. In addition, there is a set of 50 points for which the predicted value is constant while the true value varies, forming a horizontal line in Fig. 2. Upon further investigation, all of the points forming the horizontal line are associated with *u *=* *0. When *u* goes to zero after the fuel and air flow rates have reached steady-state, then $\lambda \u0302$ will also go to zero and a feasible steady-state is reached. However, if *u* drops to zero before steady-state is reached, the transient conditions cause constraint violation, which results into a nonzero Lagrange multiplier. In its current implementation, the Koopman model cannot distinguish if steady-state has been reached, it cannot distinguish these two scenarios, and a group of erroneous predictions forms.

*x*is augmented with past information. Hence the new state vector is defined by

_{k}*h*is the number of historical states included. The observables are therefore given by:

The results for this new set of observable functions with *h *=* *1 are summarized in Fig. 3, where the RMS error is lowered to 10.1, an improvement of about 7% compared to the previous set of basis functions. While the overall agreement is improved, a group of erroneous predictions is still present, as shown in Fig. 3. However, the number of points in the grouping is only 20, substantially fewer than the 50 points in the grouping when historical states are not used. The main drawback of including historical data in the state is the increased complexity of the least-squares problem solved to calibrate the functions. Specifically, Calibrating the hyperparameters for the case where *h *=* *1 took five times longer on the same computer platform, and converged to 1 and 4 for *p _{m}* and

*p*, respectively.

_{g}*flat*conditions. An error distribution analysis is shown in Fig. 4 where the percentage error is calculated as

where $\lambda \xaf$ is the average Lagrange multiplier determined in simulation. While the mean of the two distribution is similar, including the historical states reduces the standard deviation from 10.4 to 9.6 and decreases the average absolute error from 6.8% to 6.2%. However, due to the significant increase in computation time observed for the hyperparameters calibration when extending from *h *=* *0 to *h *=* *1, additional historical data would not be considered in this paper. Finally, both approaches have 10 instances where the predicted Lagrange multipliers are negative, which is not possible. However, no correction is used to bring those predictions to 0, which is another potential area of improvement.

### 4.3 Integration With Slow Dynamics Optimization.

where $\gamma 4s$ is a penalty on the Lagrange multipliers to enforce $\lambda \u0302=0$ as a soft constraint. Because the effect of the fast dynamics constraint is included in the Koopman linear system, the hierarchical optimization problem no longer requires iteration.

## 5 Simulation Study

### 5.1 Case Studies.

Two simulation cases are considered in this paper to evaluate the performance of the Koopman-Augmented MPC. Both test cases are characterized by a sharp acceleration, which causes a rapid increase in the fuel demand which will likely to lead to a constraint violation due to a limitation in the fast dynamic. The first case study represents a scenario where the ego-vehicle is following a slow lead vehicle while approaching a traffic light. Before approaching the intersection, the lead-vehicle changes lane allowing the ego-vehicle to pass while green, if an appropriate speed can be reached. The second case study examines the scenario in which the ego-vehicle is cruising at highway speed, when another vehicle enters the lane directly in front without maintaining a safety distance. The truck is forced to slow down to reestablish a safety distance and then accelerate back to highway speed in order to meet the desired overall driving distance provided by a supervisory controller.

### 5.2 Results.

The slow dynamic MPC augmented with the Koopman model for the Lagrange multipliers is applied to both case studies and compared against the centralized and iterative MPC that were developed and presented in Ref. [25]. It is worth noting that, if perfect prediction is available, $\gamma 4s$ can be chosen arbitrarily large as it only affect unfeasible control sequences and it is zero elsewhere. However, as shown in Fig. 4, there are significant uncertainties associated with the prediction of $\lambda \u0302$. For the simulations in this paper, $\gamma 4s=1\xd710\u22128$ is used.

where *m _{f}* is the total fuel over the maneuver, KE is the kinetic energy at the end of the maneuver, the subscript

*c*refers to the centralized controller and LHV is the lower heating value of the fuel. While the centralized controller achieves better overall performance, the Koopman-augmented controller approximates closely the control input profile and resulting state trajectories obtained by the iterative approach.

Central. | Iter. | Koopman | ||
---|---|---|---|---|

Fuel | (g) | 179.4 | 184.4 | 181.8 |

Savings | (%) | — | −2.79 | −1.34 |

Adj. savings | (%) | — | −2.26 | −1.30 |

Trip time | (s) | 46.18 | 44.40 | 44.57 |

Savings | (%) | — | 3.86 | 3.50 |

Distance | (m) | 600.1 | 600.0 | 600.1 |

Central. | Iter. | Koopman | ||
---|---|---|---|---|

Fuel | (g) | 179.4 | 184.4 | 181.8 |

Savings | (%) | — | −2.79 | −1.34 |

Adj. savings | (%) | — | −2.26 | −1.30 |

Trip time | (s) | 46.18 | 44.40 | 44.57 |

Savings | (%) | — | 3.86 | 3.50 |

Distance | (m) | 600.1 | 600.0 | 600.1 |

A similar comparison is performed for the second case study and the results are shown in Fig. 6.

In this case, the engine torque command drops to zero in response to the vehicle cut-in as shown in Figs. 6(a) and 6(b). Once a safe distance is established, the ego vehicle velocity has dropped from 25 m/s to 22 m/s, as shown in Fig. 6(c). The engine torque command increases to reach the desired speed. Similarly to the previous case, the centralized torque command has a chattering profile due to the different time scales and to benefit from a pulse and glide strategy. Moreover, the engine torque prescribed by the centralized controller is lower than the other two approaches resulting in a longer acceleration time, which results in a longer travel time as shown in Table 2. Again, all controllers meet the fuel air ratio (FAR) constraints as shown in Fig. 6(d).

Central. | Iter. | Koopman | ||
---|---|---|---|---|

Fuel | (g) | 342.6 | 348.1 | 344.8 |

Savings | (%) | — | −1.62 | −0.66 |

Adj. savings | (%) | — | −1.27 | −0.60 |

Trip time | (s) | 63.21 | 62.94 | 62.99 |

Savings | (%) | — | 0.43 | 0.35 |

Distance | (m) | 1500 | 1500 | 1500 |

Central. | Iter. | Koopman | ||
---|---|---|---|---|

Fuel | (g) | 342.6 | 348.1 | 344.8 |

Savings | (%) | — | −1.62 | −0.66 |

Adj. savings | (%) | — | −1.27 | −0.60 |

Trip time | (s) | 63.21 | 62.94 | 62.99 |

Savings | (%) | — | 0.43 | 0.35 |

Distance | (m) | 1500 | 1500 | 1500 |

## 6 Conclusions

This paper presented a data driven method for the prediction of the Lagrange multipliers associated with the fast dynamic optimization problem, which eliminates the requirement for iterations in a hierarchical iterative MPC for multitime-scale systems. The proposed method is based on an extension of the Koopman theory that interprets the Lagrange multipliers as an unknown nonlinear dynamical system. In this context, this paper presents the extension of the current Koopman method used in conjunction with EDMD to the prediction of the Lagrange multipliers and its integration with a slow dynamic optimization problem. Once the slow dynamic MPC was augmented the resulting Koopman linear model, the Koopman-based MPC was compared against centralized, hierarchical and iterative strategies. While the calibration process for identifying the Koopman linear system was computationally expensive and results show non-negligible uncertainties in the prediction, when integrated with the MPC, the new controller provided near optimal performance with significantly lower computational requirements compared to the initial iterative approach.

## Funding Data

This research was funded by the ARPA-E NEXTCAR Program, Project (Award No. DEAR0000801; Funder ID: 10.13039/100009224).

## References

**64**, pp.

**17**(5), pp.