Abstract
We present a tensor-based method for model selection which identifies the unknown partial differential equation that governs a dynamical system using only spatiotemporal measurements. The method circumvents a disadvantage of standard matrix-based methods which typically have large storage consumption. Using a recently developed multidimensional approximation of nonlinear dynamical systems (MANDy), we collect the nonlinear and partial derivative terms of the measured data and construct a low-rank dictionary tensor in the tensor-train (TT) format. A tensor-based linear regression problem is then built, which balances the learning accuracy, model complexity, and computational efficiency. An algebraic expression of the unknown equations can be extracted. Numerical results are demonstrated on datasets generated by the wave equation, the Burgers' equation, and a few parametric partial differential equations (PDEs).
References
1.
Johnson
,
J. B.
, and
Omland
,
K. S.
, 2004
, “
Model Selection in Ecology and Evolution
,” Trends Ecol. Evol.
,
19
(2
), pp. 101
–108
.10.1016/j.tree.2003.10.0132.
Chou
,
I.-C.
, and
Voit
,
E. O.
, 2009
, “
Recent Developments in Parameter Estimation and Structure Identification of Biochemical and Genomic Systems
,” Math. Biosci.
,
219
(2
), pp. 57
–83
.10.1016/j.mbs.2009.03.0023.
Engl
,
H. W.
,
Flamm
,
C.
,
Kügler
,
P.
,
Lu
,
J.
,
Müller
,
S.
, and
Schuster
,
P.
, 2009
, “
Inverse Problems in Systems Biology
,” Inverse Probl.
,
25
(12
), p. 123014
.10.1088/0266-5611/25/12/1230144.
Mayne
,
D. Q.
, 2014
, “
Model Predictive Control: Recent Developments and Future Promise
,” Automatica
,
50
(12
), pp. 2967
–2986
.10.1016/j.automatica.2014.10.1285.
Scher
,
S.
, and
Messori
,
G.
, 2018
, “
Predicting Weather Forecast Uncertainty With Machine Learning
,” Q. J. R. Meteorol. Soc.
,
144
(717
), pp. 2830
–2841
.10.1002/qj.34106.
Bongard
,
J.
, and
Lipson
,
H.
, 2007
, “
Automated Reverse Engineering of Nonlinear Dynamical Systems
,” Proc. Natl. Acad. Sci.
,
104
(24
), pp. 9943
–9948
.10.1073/pnas.06094761047.
Schmidt
,
M.
, and
Lipson
,
H.
, 2009
, “
Distilling Free-Form Natural Laws From Experimental Data
,” Science
,
324
(5923
), pp. 81
–85
.10.1126/science.11658938.
Raissi
,
M.
, 2018
, “
Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
,” J. Mach. Learn. Res.
,
19
, pp. 1–24.https://www.jmlr.org/papers/volume19/18-046/18-046.pdf9.
Raissi
,
M.
,
Perdikaris
,
P.
, and
Karniadakis
,
G. E.
, 2019
, “
Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations
,” J. Comput. Phys.
,
378
, pp. 686
–707
.10.1016/j.jcp.2018.10.04510.
Baydin
,
A. G.
,
Pearlmutter
,
B. A.
,
Radul
,
A. A.
, and
Siskind
,
J. M.
, 2018
, “
Automatic Differentiation in Machine Learning: A Survey
,” J. Mach. Learn. Res.
,
18
, pp. 1
–43
.https://www.jmlr.org/papers/volume18/17-468/17-468.pdf11.
Long
,
Z.
,
Lu
,
Y.
,
Ma
,
X.
, and
Dong
,
B.
, 2018
, “
PDE-Net: Learning PDEs From Data
,” The 35th International Conference on Machine Learning
, Stockholmsmässan, Stockholm, Sweden, July 10–15, pp. 3208
–3216
.https://proceedings.mlr.press/v80/long18a.html12.
Long
,
Z.
,
Lu
,
Y.
, and
Dong
,
B.
, 2019
, “
PDE-Net 2.0: Learning PDEs From Data With a Numeric-Symbolic Hybrid Deep Network
,” J. Comput. Phys.
,
399
, p. 108925
.10.1016/j.jcp.2019.10892513.
Sun
,
Y.
,
Zhang
,
L.
, and
Schaeffer
,
H.
, 2020
, “
NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data
,” The 1st Mathematical and Scientific Machine Learning Conference
, Princeton University, Princeton, NJ, July 20–24, pp. 352
–372
.https://proceedings.mlr.press/v107/sun20a.html14.
Xu
,
H.
,
Chang
,
H.
, and
Zhang
,
D.
, 2020
, “
DLGA-PDE: Discovery of PDEs With Incomplete Candidate Library Via Combination of Deep Learning and Genetic Algorithm
,” J. Comput. Phys.
,
418
, p. 109584
.10.1016/j.jcp.2020.10958415.
Chen
,
Y.
,
Luo
,
Y.
,
Liu
,
Q.
,
Xu
,
H.
, and
Zhang
,
D.
, 2022
, “
Symbolic Genetic Algorithm for Discovering Open-Form Partial Differential Equations (SGA-PDE)
,” Phys. Rev. Res.
,
4
(2
), p. 023174
.10.1103/PhysRevResearch.4.02317416.
Chen
,
P.
,
Liu
,
R.
,
Aihara
,
K.
, and
Chen
,
L.
, 2020
, “
Autoreservoir Computing for Multistep Ahead Prediction Based on the Spatiotemporal Information Transformation
,” Nat. Commun.
,
11
(1
), p. 4568
.10.1038/s41467-020-18381-017.
Wu
,
K.
, and
Xiu
,
D.
, 2020
, “
Data-Driven Deep Learning of Partial Differential Equations in Modal Space
,” J. Comput. Phys.
,
408
, p. 109307
.10.1016/j.jcp.2020.10930718.
Patel
,
R. G.
,
Trask
,
N. A.
,
Wood
,
M. A.
, and
Cyr
,
E. C.
, 2021
, “
A Physics-Informed Operator Regression Framework for Extracting Data-Driven Continuum Models
,” Comput. Methods Appl. Mech. Eng.
,
373
, p. 113500
.10.1016/j.cma.2020.11350019.
Chen
,
Z.
, and
Xiu
,
D.
, 2021
, “
On Generalized Residual Network for Deep Learning of Unknown Dynamical Systems
,” J. Comput. Phys.
,
438
, p. 110362
.10.1016/j.jcp.2021.11036220.
Brunton
,
S. L.
,
Proctor
,
J. L.
, and
Kutz
,
J. N.
, 2016
, “
Discovering Governing Equations From Data by Sparse Identification of Nonlinear Dynamical Systems
,” Proc. Natl. Acad. Sci.
,
113
(15
), pp. 3932
–3937
.10.1073/pnas.151738411321.
Rudy
,
S. H.
,
Brunton
,
S. L.
,
Proctor
,
J. L.
, and
Kutz
,
J. N.
, 2017
, “
Data-Driven Discovery of Partial Differential Equations
,” Sci. Adv.
,
3
(4
), p. e1602614
.10.1126/sciadv.160261422.
Schaeffer
,
H.
, 2017
, “
Learning Partial Differential Equations Via Data Discovery and Sparse Optimization
,” Proc. R. Soc. A
,
473
(2197
), p. 20160446
.10.1098/rspa.2016.044623.
Kaiser
,
E.
,
Kutz
,
J. N.
, and
Brunton
,
S. L.
, 2018
, “
Sparse Identification of Nonlinear Dynamics for Model Predictive Control in the Low-Data Limit
,” Proc. R. Soc. A
,
474
(2219
), p. 20180335
.10.1098/rspa.2018.033524.
Fasel
,
U.
,
Kaiser
,
E.
,
Kutz
,
J. N.
,
Brunton
,
B. W.
, and
Brunton
,
S. L.
, 2021
, “
SINDy With Control: A Tutorial
,” The 60th IEEE Conference on Decision and Control
, Austin, TX, Dec 14–17, pp. 16
–21
.10.1109/CDC45484.2021.968312025.
Mangan
,
N. M.
,
Kutz
,
J. N.
,
Brunton
,
S. L.
, and
Proctor
,
J. L.
, 2017
, “
Model Selection for Dynamical Systems Via Sparse Regression and Information Criteria
,” Proc. R. Soc. A
,
473
(2204
), p. 20170009
.10.1098/rspa.2017.000926.
Li
,
X.
,
Li
,
L.
,
Yue
,
Z.
,
Tang
,
X.
,
Voss
,
H. U.
,
Kurths
,
J.
, and
Yuan
,
Y.
, 2019
, “
Sparse Learning of Partial Differential Equations With Structured Dictionary Matrix
,” Chaos
,
29
(4
), p. 043130
.10.1063/1.505470827.
Rudy
,
S.
,
Alla
,
A.
,
Brunton
,
S. L.
, and
Kutz
,
J. N.
, 2019
, “
Data-Driven Identification of Parametric Partial Differential Equations
,” SIAM J. Appl. Dyn. Syst.
,
18
(2
), pp. 643
–660
.10.1137/18M119194428.
Ma
,
H.
,
Lu
,
X.
, and
Zhang
,
L.
, 2023
, “
Extracting Parametric Dynamics From Time-Series Data
,” Nonlinear Dyn.
,
111
(16
), pp. 15177
–15199
.10.1007/s11071-023-08643-z29.
Schaeffer
,
H.
, and
McCalla
,
S. G.
, 2017
, “
Sparse Model Selection Via Integral Terms
,” Phys. Rev. E
,
96
(2
), p. 023302
.10.1103/PhysRevE.96.02330230.
Messenger
,
D. A.
, and
Bortz
,
D. M.
, 2021
, “
Weak SINDy: Galerkin-Based Data-Driven Model Selection
,” Multiscale Model. Simul.
,
19
(3
), pp. 1474
–1497
.10.1137/20M134316631.
Schaeffer
,
H.
,
Tran
,
G.
, and
Ward
,
R.
, 2018
, “
Extracting Sparse High-Dimensional Dynamics From Limited Data
,” SIAM J. Appl. Math.
,
78
(6
), pp. 3279
–3295
.10.1137/18M116798X32.
Schaeffer
,
H.
,
Tran
,
G.
,
Ward
,
R.
, and
Zhang
,
L.
, 2020
, “
Extracting Structured Dynamical Systems Using Sparse Optimization With Very Few Samples
,” Multiscale Model. Simul.
,
18
(4
), pp. 1435
–1461
.10.1137/18M119473033.
Gelß
,
P.
,
Klus
,
S.
,
Eisert
,
J.
, and
Schütte
,
C.
, 2019
, “
Multidimensional Approximation of Nonlinear Dynamical Systems
,” ASME J. Comput. Nonlinear Dyn.
,
14
(6
), p. 061006
.10.1115/1.404314834.
Oseledets
,
I. V.
, and
Tyrtyshnikov
,
E. E.
, 2009
, “
Breaking the Curse of Dimensionality, or How to Use SVD in Many Dimensions
,” SIAM J. Sci. Comput.
,
31
(5
), pp. 3744
–3759
.10.1137/09074833035.
Oseledets
,
I. V.
, 2011
, “
Tensor-Train Decomposition
,” SIAM J. Sci. Comput.
,
33
(5
), pp. 2295
–2317
.10.1137/09075228636.
White
,
S. R.
, 1992
, “
Density Matrix Formulation for Quantum Renormalization Groups
,” Phys. Rev. Lett.
,
69
(19
), pp. 2863
–2866
.10.1103/PhysRevLett.69.286337.
Verstraete
,
F.
,
Murg
,
V.
, and
Cirac
,
J. I.
, 2008
, “
Matrix Product States, Projected Entangled Pair States, and Variational Renormalization Group Methods for Quantum Spin Systems
,” Adv. Phys.
,
57
(2
), pp. 143
–224
.10.1080/1478994080191236638.
Orús
,
R.
, 2014
, “
A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States
,” Ann. Phys.
,
349
, pp. 117
–158
.10.1016/j.aop.2014.06.01339.
Bengua
,
J. A.
,
Phien
,
H. N.
,
Tuan
,
H. D.
, and
Do
,
M. N.
, 2017
, “
Efficient Tensor Completion for Color Image and Video Recovery: Low-Rank Tensor Train
,” IEEE Trans. Image Process.
,
26
(5
), pp. 2466
–2479
.10.1109/TIP.2017.267243940.
Klus
,
S.
, and
Gelß
,
P.
, 2019
, “
Tensor-Based Algorithms for Image Classification
,” Algorithms
,
12
(11
), p. 240
.10.3390/a1211024041.
Jahnke
,
T.
, and
Huisinga
,
W.
, 2008
, “
A Dynamical Low-Rank Approach to the Chemical Master Equation
,” Bull. Math. Biol.
,
70
(8
), pp. 2283
–2302
.10.1007/s11538-008-9346-x42.
Gelß
,
P.
,
Matera
,
S.
, and
Schütte
,
C.
, 2016
, “
Solving the Master Equation Without Kinetic Monte Carlo: Tensor Train Approximations for a CO Oxidation Model
,” J. Comput. Phys.
,
314
, pp. 489
–502
.10.1016/j.jcp.2016.03.02543.
Carroll
,
J. D.
, and
Chang
,
J.-J.
, 1970
, “
Analysis of Individual Differences in Multidimensional Scaling Via an n-Way Generalization of “Eckart-Young” Decomposition
,” Psychometrika
,
35
(3
), pp. 283
–319
.10.1007/BF0231079144.
Tucker
,
L. R.
, 1963
, “
Implications of Factor Analysis of Three-Way Matrices for Measurement of Change
,” Probl. Meas. Change
,
15
, pp. 122
–137
.https://www.bibsonomy.org/bibtex/1e6c2d14ddb42cb2012afaeb8ab1d19aa/threemode45.
Tucker
,
L. R.
, 1964
, “
The Extension of Factor Analysis to Three-Dimensional Matrices
,” Contrib. Math. Psychol.
,
110119
, pp. 110
–182
.https://www.scienceopen.com/document?vid=4096120e-f38d-4e74-ad8d-c278b4da697f46.
Hackbusch
,
W.
, and
Kühn
,
S.
, 2009
, “
A New Scheme for the Tensor Representation
,” J. Fourier Anal. Appl.
,
15
(5
), pp. 706
–722
.10.1007/s00041-009-9094-947.
Lubich
,
C.
,
Rohwedder
,
T.
,
Schneider
,
R.
, and
Vandereycken
,
B.
, 2013
, “
Dynamical Approximation by Hierarchical Tucker and Tensor-Train Tensors
,” SIAM J. Matrix Anal. Appl.
,
34
(2
), pp. 470
–494
.10.1137/12088572348.
Kiers
,
H. A.
, 2000
, “
Towards a Standardized Notation and Terminology in Multiway Analysis
,” J. Chemom.
,
14
(3
), pp. 105
–122
.10.1002/1099-128X(200005/06)14:3<105::AID-CEM582>3.0.CO;2-I49.
Klus
,
S.
,
Gelß
,
P.
,
Peitz
,
S.
, and
Schütte
,
C.
, 2018
, “
Tensor-Based Dynamic Mode Decomposition
,” Nonlinearity
,
31
(7
), pp. 3359
–3380
.10.1088/1361-6544/aabc8f50.
Trefethen
,
L. N.
, 2000
, “Spectral Methods in MATLAB,” Society for Industrial and Applied Mathematics
, Philadelphia, PA.10.1137/1.978089871959851.
He
,
L.
,
Chen
,
K.
,
Xu
,
W.
,
Zhou
,
J.
, and
Wang
,
F.
, 2018
, “
Boosted Sparse and Low-Rank Tensor Regression
,” The 32nd Conference on Neural Information Processing Systems
, Montréal, QC, Canada, Dec. 2–8.https://papers.nips.cc/paper_files/paper/2018/hash/8d34201a5b85900908db6cae92723617-Abstract.html52.
Ahmed
,
T.
,
Raja
,
H.
, and
Bajwa
,
W. U.
, 2020
, “
Tensor Regression Using Low-Rank and Sparse Tucker Decompositions
,” SIAM J. Math. Data Sci.
,
2
(4
), pp. 944
–966
.10.1137/19M129933553.
Ou-Yang
,
L.
,
Zhang
,
X.-F.
, and
Yan
,
H.
, 2020
, “
Sparse Regularized Low-Rank Tensor Regression With Applications in Genomic Data Analysis
,” Pattern Recognit.
,
107
, p. 107516
.10.1016/j.patcog.2020.10751654.
Zhang
,
L.
, and
Schaeffer
,
H.
, 2019
, “
On the Convergence of the SINDy Algorithm
,” Multiscale Model. Simul.
,
17
(3
), pp. 948
–972
.10.1137/18M118982855.
Lu
,
X.
,
Ma
,
H.
, and
Zhang
,
L.
, 2023
, “
Model Selection Via Reweighted Partial Sparse Recovery
,” J. Comput. Appl. Math.
,
438
, p. 115553
.10.1016/j.cam.2023.115553
