Abstract

Fitting a specified model to data is critical in many science and engineering fields. A major task in fitting a specified model to data is to estimate the value of each parameter in the model. Iterative local methods, such as the Gauss–Newton method and the Levenberg–Marquardt method, are often employed for parameter estimation in nonlinear models. However, practitioners must guess the initial value for each parameter to initialize these iterative local methods. A poor initial guess can contribute to non-convergence of these methods or lead these methods to converge to a wrong or inferior solution. In this paper, a solution interval method is introduced to find the optimal estimator for each parameter in a nonlinear model that minimizes the squared error of the fit. To initialize this method, it is not necessary for practitioners to guess the initial value of each parameter in a nonlinear model. The method includes three algorithms that require different levels of computational power to find the optimal parameter estimators. The method constructs a solution interval for each parameter in the model. These solution intervals significantly reduce the search space for optimal parameter estimators. The method also provides an empirical probability distribution for each parameter, which is valuable for parameter uncertainty assessment. The solution interval method is validated through two case studies in which the Michaelis–Menten model and Fick’s second law are fit to experimental data sets, respectively. These case studies show that the solution interval method can find optimal parameter estimators efficiently. A four-step procedure for implementing the solution interval method in practice is also outlined.

References

1.
Bannantine
,
J.
,
Comer
,
J.
, and
Handrock
,
J.
,
1990
,
Fundamentals of Metal Fatigue Analysis
,
Prentice Hall
,
Englewood Cliffs, NJ
.
2.
Juvinall
,
R. C.
, and
Marshek
,
K. M.
,
2000
,
Fundamentals of Machine Component Design
,
John Wiley & Sons, Inc
,
New York, NY
.
3.
Wirsching
,
P. H.
,
1983
,
Statistical Summaries of Fatigue Data for Design Purposes
,
NASA Contractor Report 3697
,
Washington, DC
, https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19830021460.pdf
4.
ASTM
,
2015
,
E739-10. Standard Practice for Statistical Analysis of Linear or Linearized Stress-Life (S-N) and Strain-Life (ɛ–N) Fatigue Data
,
ASTM International
,
West Conshohocken, PA
, https://www.astm.org/Standards/E739
5.
Crank
,
J.
,
1975
,
The Mathematics of Diffusion
,
Oxford University Press
,
Oxford
.
6.
ASTM
,
2014
,
D5229/D5229M-14 Standard Test Method for Moisture Absorption Properties and Equilibrium Conditioning of Polymer Matrix Composite Materials
,
ASTM International
,
West Conshohocken, PA
, https://www.astm.org/Standards/D5229.htm
7.
McKague
,
E.
,
Reynolds
,
J.
, and
Halkias
,
J.
,
1975
,
Life Assurance of Composite Structures. Volume I. Moisture Effects
,
AFML-TR-75-51, Fort Worth Division of General Dynamics, Wright-Patterson Air Force Base
,
Ohio
, https://apps.dtic.mil/dtic/tr/fulltext/u2/b006906.pdf
8.
Jennrich
,
R. I.
, and
Ralston
,
M. L.
,
1979
, “
Fitting Nonlinear Models to Data
,”
Annu. Rev. Biophys. Bioeng.
,
8
(
1
), pp.
195
238
. 10.1146/annurev.bb.08.060179.001211
9.
Montgomery
,
D. C.
,
Peck
,
E. A.
, and
Vining
,
G. G.
,
2006
,
Introduction to Linear Regression Analysis
,
Wiley-Interscience
,
Hoboken, NJ
.
10.
Hauser
,
J. R.
,
2009
,
Numerical Methods for Nonlinear Engineering Models
,
Springer Science & Business Media
,
Berlin/Heidelberg, Germany
.
11.
Moré
,
J. J.
,
1978
,
“The Levenberg-Marquardt Algorithm: Implementation and Theory,” Numerical Analysis
,
Springer
,
Berlin/Heidelberg, Germany
, pp.
105
116
, https://link.springer.com/book/10.1007%2FBFb0067690#page=116
12.
Hansen
,
P. C.
,
Pereyra
,
V.
, and
Scherer
,
G.
,
2013
,
Least Squares Data Fitting With Applications
,
The Johns Hopkins University Press
,
Baltimore, ML
.
13.
Björck
,
Å
,
1996
,
Numerical Methods for Least Squares Problems
,
Society for Industrial and Applied Mathematics
,
Philadelphia, PA
.
14.
Gelman
,
A.
,
Carlin
,
J. B.
,
Stern
,
H. S.
,
Dunson
,
D. B.
,
Vehtari
,
A.
, and
Rubin
,
D. B.
,
2013
,
Bayesian Data Analysis
,
CRC Press
,
Boca Raton, FL
.
15.
Motulsky
,
H. J.
, and
Ransnas
,
L. A.
,
1987
, “
Fitting Curves to Data Using Nonlinear Regression: A Practical and Nonmathematical Review
,”
FASEB J.
,
1
(
5
), pp.
365
374
. 10.1096/fasebj.1.5.3315805
16.
Bates
,
D. M.
, and
Watts
,
D. G.
,
1988
,
Nonlinear Regression Analysis and Its Applications
,
John Wiley & Sons
,
New York
, https://onlinelibrary.wiley.com/doi/book/10.1002/9780470316757
17.
Fraley
,
C.
,
1988
,
Algorithms for Nonlinear Least-Squares Problems
,
SOL 88-16, Systems Optimization Laboratory
,
Stanford, CA
, https://apps.dtic.mil/dtic/tr/fulltext/u2/a201848.pdf
18.
Schaller
,
R. R.
,
1997
, “
Moore's Law: Past, Present and Future
,”
IEEE Spectrum
,
34
(
6
), pp.
52
59
. 10.1109/6.591665
19.
Press
,
W. H.
,
Teukolsky
,
S. A.
,
Vetterling
,
W. T.
, and
Flannery
,
B. P.
,
2007
,
Numerical Recipes: The Art of Scientific Computing
,
Cambridge University Press
,
Cambridge
.
20.
Ramsin
,
H.
, and
Wedin
,
P-Å
,
1977
, “
A Comparison of Some Algorithms for the Nonlinear Least Squares Problem
,”
BIT Numer. Math.
,
17
(
1
), pp.
72
90
. 10.1007/BF01932400
21.
Gill
,
P. E.
, and
Murray
,
W.
,
1976
,
“Nonlinear Least Squares and Nonlinearly Constrained Optimization,” Numerical Analysis
,
G. A.
Watson
, ed.,
Springer-Verlag Berlin
,
Germany
,
134
147
22.
Dennis
,
J. E.
, Jr
, and
Schnabel
,
R. B.
,
1996
,
Numerical Methods for Unconstrained Optimization and Nonlinear Equations
,
Society for Industrial and Applied Mathematics
,
Philadelphia, PA
.
23.
Gill
,
P. E.
, and
Murray
,
W.
,
1978
, “
Algorithms for the Solution of the Nonlinear Least-Squares Problem
,”
SIAM J. Numer. Anal.
,
15
(
5
), pp.
977
992
. 10.1137/0715063
24.
Dennis
,
J. E.
, Jr
,
Gay
,
D. M.
, and
Welsch
,
R. E.
,
1981
, “
An Adaptive Nonlinear Least Square Algorithm
,”
ACM Trans. Math. Software
,
7
(
3
), pp.
348
368
. 10.1145/355958.355965
25.
Levenberg
,
K.
,
1944
, “
A Method for the Solution of Certain Non-Linear Problems in Least Squares
,”
Quarterly Appl. Math.
,
2
(
2
), pp.
164
168
. 10.1090/qam/10666
26.
Marquardt
,
D. W.
,
1963
, “
An Algorithm for Least-Squares Estimation of Nonlinear Parameters
,”
J. Soc. Ind. Appl. Math.
,
11
(
2
), pp.
431
441
. 10.1137/0111030
27.
Moré
,
J. J.
,
1983
,
“Recent Developments in Algorithms and Software for Trust Region Methods,” Mathematical Programming The State of the Art
,
Springer-Verlag, Berlin
,
Germany
,
258
287
, https://link.springer.com/chapter/10.1007/978-3-642-68874-4_11#citeas
28.
Fletcher
,
R.
,
1987
,
Practical Methods of Optimization
,
John Wiley & Sons
,
Chichester
.
29.
Coleman
,
T. F.
, and
Li
,
Y.
,
1994
, “
On the Convergence of Interior-Reflective Newton Methods for Nonlinear Minimization Subject to Bounds
,”
Math. Program.
,
67
(
1–3
), pp.
189
224
. 10.1007/BF01582221
30.
Conn
,
A. R.
,
Gould
,
N. I.
, and
Toint
,
P. L.
,
2000
,
Trust Region Methods
,
Society for Industrial and Applied Mathematics and Mathematical Programming Society
,
Philadelphia, PA
.
31.
Yuan
,
Y.-x.
,
2015
, “
Recent Advances in Trust Region Algorithms
,”
Math. Program.
,
151
(
1
), pp.
249
281
. 10.1007/s10107-015-0893-2
32.
Sobieszczanski-Sobieski
,
J.
,
Morris
,
A.
, and
VanTooren
,
M.
,
2015
,
Multidisciplinary Design Optimization Supported by Knowledge Based Engineering
,
John Wiley & Sons, The Atrium
,
Southern Gate, Chichester
.
33.
Chang
,
W.-D.
,
2007
, “
Nonlinear System Identification and Control Using A Real-Coded Genetic Algorithm
,”
Appl. Math. Model.
,
31
(
3
), pp.
541
550
. 10.1016/j.apm.2005.11.024
34.
Mohan
,
S.
,
1997
, “
Parameter Estimation of Nonlinear Muskingum Models Using Genetic Algorithm
,”
J. Hydraul. Eng.
,
123
(
2
), pp.
137
142
. 10.1061/(ASCE)0733-9429(1997)123:2(137)
35.
Wong
,
K.-P.
,
Meikle
,
S. R.
,
Feng
,
D.
, and
Fulham
,
M. J.
,
2002
, “
Estimation of Input Function and Kinetic Parameters Using Simulated Annealing: Application in a Flow Model
,”
IEEE Trans. Nucl. Sci.
,
49
(
3
), pp.
707
713
. 10.1109/TNS.2002.1039552
36.
Schwaab
,
M.
,
Biscaia
,
E. C.
, Jr
,
Monteiro
,
J. L.
, and
Pinto
,
J. C.
,
2008
, “
Nonlinear Parameter Estimation Through Particle Swarm Optimization
,”
Chem. Eng. Sci.
,
63
(
6
), pp.
1542
1552
. 10.1016/j.ces.2007.11.024
37.
Tierney
,
L.
, and
Kadane
,
J. B.
,
1986
, “
Accurate Approximations for Posterior Moments and Marginal Densities
,”
J. Am. Stat. Assoc.
,
81
(
393
), pp.
82
86
. 10.1080/01621459.1986.10478240
38.
Gilks
,
W. R.
,
Richardson
,
S.
, and
Spiegelhalter
,
D.
,
1996
,
Markov Chain Monte Carlo in Practice
,
Chapman and Hall/CRC
,
Boca Raton, FL
.
39.
Congdon
,
P.
,
2014
,
Applied Bayesian Modelling
,
John Wiley & Sons
,
The Atrium, Southern Gate, Chichester
.
40.
Motulsky
,
H.
, and
Christopoulos
,
A.
,
2004
,
Fitting Models to Biological Data Using Linear and Nonlinear Regression: A Practical Guide to Curve Fitting
,
Oxford University Press
,
Oxford
.
41.
Archontoulis
,
S. V.
, and
Miguez
,
F. E.
,
2015
, “
Nonlinear Regression Models and Applications in Agricultural Research
,”
Agron. J.
,
107
(
2
), pp.
786
798
. 10.2134/agronj2012.0506
42.
Garthwaite
,
P. H.
,
Kadane
,
J. B.
, and
O'Hagan
,
A.
,
2005
, “
Statistical Methods for Eliciting Probability Distributions
,”
J. Am. Stat. Assoc.
,
100
(
470
), pp.
680
701
. 10.1198/016214505000000105
43.
Johnson
,
K. A.
, and
Goody
,
R. S.
,
2011
, “
The Original Michaelis Constant: Translation of the 1913 Michaelis–Menten Paper
,”
Biochemistry
,
50
(
39
), pp.
8264
8269
. 10.1021/bi201284u
44.
Voet
,
D.
, and
Voet
,
J. G.
,
2010
,
Biochemistry
,
John Wiley & Sons
,
Hoboken, NJ
.
45.
Atkins
,
G.
, and
Nimmo
,
I.
,
1975
, “
A Comparison of Seven Methods for Fitting the Michaelis-Menten Equation
,”
Biochem. J.
,
149
(
3
), pp.
775
777
. 10.1042/bj1490775
46.
Jost
,
W.
,
1952
,
Diffusion in Solids, Liquids, Gases
,
Academic Press
,
New York
.
47.
Shen
,
C.-H.
, and
Springer
,
G. S.
,
1976
, “
Moisture Absorption and Desorption of Composite Materials
,”
J. Compos. Mater.
,
10
(
1
), pp.
2
20
. 10.1177/002199837601000101
48.
Fan
,
Y.
,
Gomez
,
A.
,
Ferraro
,
S.
,
Pinto
,
B.
,
Muliana
,
A.
, and
La Saponara
,
V.
,
2017
, “
The Effects of Temperatures and Volumetric Expansion on the Diffusion of Fluids Through Solid Polymers
,”
J. Appl. Polym. Sci.
,
134
(
31
), p.
45151
. 10.1002/app.45151
49.
ISO
,
2008
,
62 Plastics—Determination of Water Absorption
,
ISO
,
Geneva, Switzerland
, https://www.iso.org/standard/41672.html
50.
Brent
,
R. P.
,
1973
,
Algorithms for Minimization Without Derivatives
,
Prentice-Hall, Englewood Cliffs
,
New Jersey
.
You do not currently have access to this content.