The major drawback of the Bayesian approach to model calibration is the computational burden involved in describing the posterior distribution of the unknown model parameters arising from the fact that typical Markov chain Monte Carlo (MCMC) samplers require thousands of forward model evaluations. In this work, we develop a variational Bayesian approach to model calibration which uses an information theoretic criterion to recast the posterior problem as an optimization problem. Specifically, we parameterize the posterior using the family of Gaussian mixtures and seek to minimize the information loss incurred by replacing the true posterior with an approximate one. Our approach is of particular importance in underdetermined problems with expensive forward models in which both the classical approach of minimizing a potentially regularized misfit function and MCMC are not viable options. We test our methodology on two surrogate-free examples and show that it dramatically outperforms MCMC methods.

References

References
1.
Oliver
,
D. S.
,
Reynolds
,
A. C.
, and
Liu
,
N.
,
2008
,
Inverse Theory for Petroleum Reservoir Characterization and History Matching
,
Cambridge University Press
,
Cambridge, UK
.
2.
Tarantola
,
A.
,
2005
,
Inverse Problem Theory and Methods for Model Parameter Estimation
,
Society for Industrial and Applied Mathematics
,
Philadelphia, PA
.
3.
Jaynes
,
E. T.
, and
Bretthorst
,
G. L.
,
2003
,
Probability Theory: The Logic of Science
,
Cambridge University Press
,
Cambridge, UK
.
4.
Bilionis
,
I.
,
Drewniak
,
B. A.
, and
Constantinescu
,
E. M.
,
2015
, “
Crop Physiology Calibration in the CLM
,”
Geosci. Model Dev.
,
8
(
4
), pp.
1071
1083
.
5.
Kalnay
,
E.
,
2002
,
Atmospheric Modeling, Data Assimilation and Predictability
,
1st ed.
,
Cambridge University Press
,
Cambridge, UK
.
6.
Navon
,
I. M.
,
2009
, “
Data Assimilation for Numerical Weather Prediction: A Review
,”
Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications
,
K. P.
Seon
and
L.
Xu
, eds.,
Springer
,
Berlin
, pp.
21
65
.
7.
Hill
,
M. C.
, and
Tiedeman
,
C. R.
,
2006
,
Effective Groundwater Model Calibration: With Analysis of Data, Sensitivities, Predictions, and Uncertainty
,
Wiley
,
New York
.
8.
Tikhonov
,
A. N.
,
1963
, “
Regularization of Incorrectly Formulated Problems and the Regularization
,”
Dokl. Akad. Nauk SSSR
,
151
, pp.
501
504
.
9.
Fichtner
,
A.
,
2010
,
Full Seismic Waveform Modelling and Inversion
(Advances in Geophysical and Environmental Mechanics and Mathematics),
Springer
,
Berlin
.
10.
Metropolis
,
N.
,
Rosenbluth
,
A. W.
,
Rosenbluth
,
M. N.
,
Teller
,
A. H.
, and
Teller
,
E.
,
1953
, “
Equation of State Calculations by Fast Computing Machines
,”
J. Chem. Phys.
,
21
(
6
), pp.
1087
1092
.
11.
Hastings
,
W. K.
,
1970
, “
Monte Carlo Sampling Methods Using Markov Chains and Their Applications
,”
Biometrika
,
57
(
1
), pp.
97
109
.
12.
Haario
,
H.
,
Saksman
,
E.
, and
Tamminen
,
J.
,
2001
, “
An Adaptive Metropolis Algorithm
,”
Bernoulli
,
7
(
2
), pp.
223
242
.
13.
Mira
,
A.
,
2001
, “
On Metropolis–Hastings Algorithms With Delayed Rejection
,”
Metron
,
59
(
3–4
), pp.
231
241
.
14.
Tierney
,
L.
, and
Antonietta
,
M.
,
1999
, “
Some Adaptive Monte Carlo Methods for Bayesian Inference
,”
Stat. Med.
,
18
(
17–18
), pp.
2507
2515
.
15.
Haario
,
H.
,
Laine
,
M.
,
Mira
,
A.
, and
Saksman
,
E.
,
2006
, “
DRAM: Efficient Adaptive MCMC
,”
Stat. Comput.
,
16
(
4
), pp.
339
354
.
16.
Roberts
,
G. O.
, and
Rosenthal
,
J. S.
,
1998
, “
Optimal Scaling of Discrete Approximations to Langevin Diffusions
,”
J. R. Stat. Soc. Ser. B
,
60
(
1
), pp.
255
268
.
17.
Duane
,
S.
,
Kennedy
,
A. D.
,
Pendleton
,
B. J.
, and
Roweth
,
D.
,
1987
, “
Hybrid Monte Carlo
,”
Phys. Lett. B
,
195
(
2
), pp.
216
222
.
18.
Girolami
,
M.
, and
Calderhead
,
B.
,
2011
, “
Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods
,”
J. R. Stat. Soc. Ser. B
,
73
(
2
), pp.
123
214
.
19.
Parno
,
M. D.
,
2015
, “
Transport Maps for Accelerated Bayesian Computation
,”
Ph.D. thesis
, Massachusetts Institute of Technology, Cambridge, MA.
20.
Kennedy
,
M. C.
, and
O'Hagan
,
A.
,
2001
, “
Bayesian Calibration of Computer Models
,”
J. R. Stat. Soc. Ser. B
,
63
(
3
), pp.
425
464
.
21.
Ghanem
,
R. G.
, and
Spanos
,
P. D.
,
2003
,
Stochastic Finite Elements: A Spectral Approach
,
Dover Publications Inc.
,
Mineola, NY
.
22.
Xiu
,
D.
, and
Karniadakis
,
G. E.
,
2002
, “
The Wiener–Askey Polynomial Chaos for Stochastic Differential Equations
,”
SIAM J. Sci. Comput.
,
24
(
2
), pp.
619
644
.
23.
Marzouk
,
Y.
, and
Xiu
,
D.
,
2009
, “
A Stochastic Collocation Approach to Bayesian Inference in Inverse Problems
,”
Commun. Comput. Phys.
,
6
(
4
), pp.
826
847
.
24.
Marzouk
,
Y. M.
,
Najm
,
H. N.
, and
Rahn
,
L. A.
,
2007
, “
Stochastic Spectral Methods for Efficient Bayesian Solution of Inverse Problems
,”
J. Comput. Phys.
,
224
(
2
), pp.
560
586
.
25.
Marzouk
,
Y. M.
, and
Najm
,
H. N.
,
2009
, “
Dimensionality Reduction and Polynomial Chaos Acceleration of Bayesian Inference in Inverse Problems
,”
J. Comput. Phys.
,
228
(
6
), pp.
1862
1902
.
26.
Bilionis
,
I.
, and
Zabaras
,
N.
,
2014
, “
Solution of Inverse Problems With Limited Forward Solver Evaluations: A Bayesian Perspective
,”
Inverse Probl.
,
30
(
1
), p.
015004
.
27.
Ghosh
,
J. K.
,
Delampady
,
M.
, and
Samanta
,
T.
,
2007
,
An Introduction to Bayesian Analysis: Theory and Methods
,
Springer Science & Business Media
,
Heidelberg, Germany
.
28.
Griewank
,
A.
, and
Walther
,
A.
,
2008
,
Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation
,
2nd ed.
,
Society for Industrial and Applied Mathematics
,
Philadelphia, PA
.
29.
Plessix
,
R.-E.
,
2006
, “
A Review of the Adjoint-State Method for Computing the Gradient of a Functional With Geophysical Applications
,”
Geophys. J. Int.
,
167
(
2
), pp.
495
503
.
30.
Fox
,
C. W.
, and
Roberts
,
S. J.
,
2011
, “
A Tutorial on Variational Bayesian Inference
,”
Artif. Intell. Rev.
,
38
(
2
), pp.
85
95
.
31.
Chen
,
P.
,
Zabaras
,
N.
, and
Bilionis
,
I.
,
2015
, “
Uncertainty Propagation Using Infinite Mixture of Gaussian Processes and Variational Bayesian Inference
,”
J. Comput. Phys.
,
284
, pp.
291
333
.
32.
Ormerod
,
J. T.
, and
Wand
,
M. P.
,
2010
, “
Explaining Variational Approximations
,”
Am. Stat.
,
64
(
2
), pp.
140
153
.
33.
Li
,
L.
,
Silva
,
J.
,
Zhou
,
M.
, and
Carin
,
L.
,
2012
, “
Online Bayesian Dictionary Learning for Large Datasets
,”
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
, Kyoto, Japan, Mar. 25–30, pp.
2157
2160
.
34.
Blei
,
D. M.
,
Ng
,
A. Y.
, and
Jordan
,
M. I.
,
2003
, “
Latent Dirichlet Allocation
,”
J. Mach. Learn. Res.
,
3
, pp.
993
1022
.
35.
Hoffman
,
M. D.
,
Blei
,
D. M.
, and
Bach
,
F. R.
,
2010
, “
Online Learning for Latent Dirichlet Allocation
,”
Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems
, Vancouver, BC, Canada, Dec. 6–9, pp.
856
864
.
36.
Jin
,
B.
,
2012
, “
A Variational Bayesian Method to Inverse Problems With Impulsive Noise
,”
J. Comput. Phys.
,
231
(
2
), pp.
423
435
.
37.
Franck
,
I. M.
, and
Koutsourelakis
,
P. S.
,
2016
, “
Sparse Variational Bayesian Approximations for Nonlinear Inverse Problems: Applications in Nonlinear Elastography
,”
Comput. Methods Appl. Mech. Eng.
,
299
, pp.
215
244
.
38.
Pinski
,
F.
,
Simpson
,
G.
,
Stuart
,
A.
, and
Weber
,
H.
,
2015
, “
Kullback–Leibler Approximation for Probability Measures on Infinite Dimensional Spaces
,”
SIAM J. Math. Anal.
,
47
(
6
), pp.
4091
4122
.
39.
Pinski
,
F. J.
,
Simpson
,
G.
,
Stuart
,
A. M.
, and
Weber
,
H.
,
2015
, “
Algorithms for Kullback–Leibler Approximation of Probability Measures in Infinite Dimensions
,”
SIAM J. Sci. Comput.
,
37
(
6
), pp.
A2733
A2757
.
40.
Gershman
,
S.
,
Hoffman
,
M. D.
, and
Blei
,
D. M.
,
2012
, “
Nonparametric Variational Inference
,”
Proceedings of the 29th International Conference on Machine Learning
, Edinburgh, Scotland, UK.
41.
McLachlan
,
G.
, and
Peel
,
D.
,
2004
,
Finite Mixture Models
,
Wiley
,
New York
.
42.
Jordan
,
M. I.
,
Ghahramani
,
Z.
,
Zaakkola
,
T. S.
, and
Saul
,
L. K.
,
1999
, “
An Introduction to Variational Methods for Graphical Models
,”
Mach. Learn.
,
37
(
2
), pp.
183
233
.
43.
Kullback
,
S.
, and
Leibler
,
R. A.
,
1951
, “
On Information and Sufficiency
,”
Ann. Math. Stat.
,
22
(
1
), pp.
79
86
.
44.
Robbins
,
H.
, and
Monro
,
S.
,
1951
, “
A Stochastic Approximation Method
,”
Ann. Math. Stat.
,
22
(
3
), pp.
400
407
.
45.
Bilionis
,
I.
, and
Zabaras
,
N.
,
2013
, “
A Stochastic Optimization Approach to Coarse-Graining Using a Relative-Entropy Framework
,”
J. Chem. Phys.
,
138
(
4
), p.
044313
.
46.
Bilionis
,
I.
, and
Koutsourelakis
,
P. S.
,
2012
, “
Free Energy Computations by Minimization of Kullback–Leibler Divergence: An Efficient Adaptive Biasing Potential Method for Sparse Representations
,”
J. Comput. Phys.
,
231
(
9
), pp.
3849
3870
.
47.
Huber
,
M. F.
,
Bailey
,
T.
,
Durrant-Whyte
,
H.
, and
Hanebeck
,
U. D.
,
2008
, “
On Entropy Approximation for Gaussian Mixture Random Vectors
,”
IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI)
, Seoul, South Korea, Aug. 20–22, pp.
181
188
.
48.
Bacharoglou
,
A. G.
,
2010
, “
Approximation of Probability Distributions by Convex Mixtures of Gaussian Measures
,”
Proc. Am. Math. Soc.
,
138
(
7
), pp.
2619
2628
.
49.
Byrd
,
R. H.
,
Lu
,
P.
,
Nocedal
,
J.
, and
Zhu
,
C.
,
1995
, “
A Limited Memory Algorithm for Bound Constrained Optimization
,”
SIAM J. Sci. Comput.
,
16
(
5
), pp.
1190
1208
.
50.
Atchadé
,
Y. F.
,
2006
, “
An Adaptive Version for the Metropolis Adjusted Langevin Algorithm With a Truncated Drift
,”
Methodol. Comput. Appl. Probab.
,
8
(
2
), pp.
235
254
.
51.
Katsounaros
,
I.
,
Dortsiou
,
M.
,
Polatides
,
C.
,
Preston
,
S.
,
Kypraios
,
T.
, and
Kyriacou
,
G.
,
2012
, “
Reaction Pathways in the Electrochemical Reduction of Nitrate on Tin
,”
Electrochim. Acta
,
71
, pp.
270
276
.
52.
Guyer
,
J. E.
,
Wheeler
,
D.
, and
Warren
,
J. A.
,
2009
, “
FiPy: Partial Differential Equations With Python
,”
Comput. Sci. Eng.
,
11
(
3
), pp.
6
15
.
You do not currently have access to this content.