An adjoint sensitivity-based approach to determine the gradient and Hessian of cost functions for system identification of dynamical systems is presented. The motivation is the development of a computationally efficient approach relative to the direct differentiation (DD) technique and which overcomes the challenges of the step-size selection in finite difference (FD) approaches. An optimization framework is used to determine the parameters of a dynamical system which minimizes a summation of a scalar cost function evaluated at the discrete measurement instants. The discrete time measurements result in discontinuities in the Lagrange multipliers. Two approaches labeled as the Adjoint and the Hybrid are developed for the calculation of the gradient and Hessian for gradient-based optimization algorithms. The proposed approach is illustrated on the Lorenz 63 model where part of the initial conditions and model parameters are estimated using synthetic data. Examples of identifying model parameters of light curves of type 1a supernovae and a two-tank dynamic model using publicly available data are also included.

References

References
1.
Raffard
,
R. L.
, and
Tomlin
,
C. J.
,
2005
, “
Second Order Adjoint-Based Optimization of Ordinary and Partial Differential Equations With Application to Air Traffic Flow
,”
American Control Conference
(
ACC
), Portland, OR, June 8–10, pp.
798
803
.
2.
Özyurt
,
D. B.
, and
Barton
,
P. I.
,
2005
, “
Cheap Second Order Directional Derivatives of Stiff Ode Embedded Functionals
,”
SIAM J. Sci. Comput.
,
26
(
5
), pp.
1725
1743
.
3.
Nocedal
,
J.
, and
Wright
,
S.
,
2006
,
Numerical Optimization
,
Springer Science & Business Media
, New York.
4.
Kelley
,
C. T.
,
1999
,
Iterative Methods for Optimization
, Vol.
18
,
SIAM
, Philadelphia, PA.
5.
Cao
,
Y.
,
Li
,
S.
,
Petzold
,
L.
, and
Serban
,
R.
,
2003
, “
Adjoint Sensitivity Analysis for Differential-Algebraic Equations: The Adjoint Dae System and Its Numerical Solution
,”
SIAM J. Sci. Comput.
,
24
(
3
), pp.
1076
1089
.
6.
Cao
,
Y.
,
Li
,
S.
, and
Petzold
,
L.
,
2002
, “
Adjoint Sensitivity Analysis for Differential-Algebraic Equations: Algorithms and Software
,”
J. Comput. Appl. Math.
,
149
(
1
), pp.
171
191
.
7.
Sengupta
,
B.
,
Friston
,
K. J.
, and
Penny
,
W. D.
,
2014
, “
Efficient Gradient Computation for Dynamical Models
,”
NeuroImage
,
98
, pp.
521
527
.
8.
Raffard
,
R. L.
,
Amonlirdviman
,
K.
,
Axelrod
,
J. D.
, and
Tomlin
,
C. J.
,
2006
, “
Parameter Identification Via the Adjoint Method: Application to Protein Regulatory Networks
,”
IFAC Proc.
,
39
(
2
), pp.
475
482
.
9.
Raffard
,
R. L.
,
Amonlirdviman
,
K.
,
Axelrod
,
J. D.
, and
Tomlin
,
C. J.
,
2008
, “
An Adjoint-Based Parameter Identification Algorithm Applied to Planar Cell Polarity Signaling
,”
IEEE Trans. Autom. Control
, 53 (Special Issue), pp.
109
121
.
10.
Suwartadi
,
E.
,
Krogstad
,
S.
, and
Foss
,
B.
,
2009
, “
On State Constraints of Adjoint Optimization in Oil Reservoir Water-Flooding
,”
SPE/EAGE Reservoir Characterization & Simulation Conference
, Abu Dhabi, United Arab Emirates, Oct. 19–21.https://www.researchgate.net/profile/Eka_Suwartadi/publication/254530264_On_State_Constraints_of_Adjoint_Optimization_in_Oil_Reservoir_Waterflooding/links/00b7d53c3e3e78a46f000000/On-State-Constraints-of-Adjoint-Optimization-in-Oil-Reservoir-Waterflooding.pdf
11.
Le Dimet
,
F.-X.
,
Navon
,
I. M.
, and
Daescu
,
D. N.
,
2002
, “
Second-Order Information in Data Assimilation
,”
Mon. Weather Rev.
,
130
(
3
), pp.
629
648
.
12.
Wang
,
Z.
,
Navon
,
I. M.
,
Le Dimet
,
F.
, and
Zou
,
X.
,
1992
, “
The Second Order Adjoint Analysis: Theory and Applications
,”
Meteorol. Atmos. Phys.
,
50
(
1–3
), pp.
3
20
.
13.
Sandu
,
A.
,
Daescu
,
D. N.
, and
Carmichael
,
G. R.
,
2003
, “
Direct and Adjoint Sensitivity Analysis of Chemical Kinetic Systems With Kpp—Part I: Theory and Software Tools
,”
Atmos. Environ.
,
37
(
36
), pp.
5083
5096
.
14.
Shichitake
,
M.
, and
Kawahara
,
M.
,
2008
, “
Optimal Control Applied to Water Flow Using Second Order Adjoint Method
,”
Int. J. Comput. Fluid Dyn.
,
22
(
5
), pp.
351
365
.
15.
Liu
,
S.
, and
Bewley
,
T. R.
,
2003
, “
Adjoint-Based System Identification and Feedforward Control Optimization in Automotive Powertrain Subsystems
,”
American Control Conference
, (
ACC
), Denver, CO, June 4–6, pp.
2566
2571
.
16.
Nandi
,
S.
, and
Singh
,
T.
,
2017
, “
Adjoint Based Hessians for Optimization Problems in System Identification
,”
IEEE Conference on Control Technology and Applications
(
CCTA
), Mauna Lani, HI, Aug. 27–30, pp.
626
631
.
17.
Griewank
,
A.
, and
Walther
,
A.
,
2008
,
Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation
,
SIAM
, Philadelphia, PA.
18.
Alexe
,
M.
, and
Sandu
,
A.
,
2009
, “
Forward and Adjoint Sensitivity Analysis With Continuous Explicit Runge–Kutta Schemes
,”
Appl. Math. Comput.
,
208
(
2
), pp.
328
346
.
19.
Lu
,
F.
,
Xu
,
D.
, and
Wen
,
G.
,
2004
, “
Estimation of Initial Conditions and Parameters of a Chaotic Evolution Process From a Short Time Series
,”
Chaos: Interdiscip. J. Nonlinear Sci.
,
14
(
4
), pp.
1050
1055
.
20.
Lazzús
,
J. A.
,
Rivera
,
M.
, and
López-Caraballo
,
C. H.
,
2016
, “
Parameter Estimation of Lorenz Chaotic System Using a Hybrid Swarm Intelligence Algorithm
,”
Phys. Lett. A
,
380
(
11–12
), pp.
1164
1171
.
21.
Rust
,
B. W.
,
Oleary
,
D. P.
, and
Mullen
,
K. M.
,
2010
, “
Modelling Type 1a Supernova Light Curves
,” Exponential Data Fitting and Its Applications, Bentham Science Publishers, Emirate of Sharjah, United Arab Emirates, pp.
169
186
.
22.
Wigren, T., and Schoukens, J., 2013, “Three Free Data Sets for Development and Benchmarking in Nonlinear System Identification,”
European Control Conference
, Zurich, Switzerland, July 17–19, pp. 2933–2938. https://ieeexplore.ieee.org/abstract/document/6669201/
You do not currently have access to this content.