×

Learning on dynamic statistical manifolds. (English) Zbl 1472.62011

Summary: Hyperbolic balance laws with uncertain (random) parameters and inputs are ubiquitous in science and engineering. Quantification of uncertainty in predictions derived from such laws, and reduction of predictive uncertainty via data assimilation, remain an open challenge. That is due to nonlinearity of governing equations, whose solutions are highly non-Gaussian and often discontinuous. To ameliorate these issues in a computationally efficient way, we use the method of distributions, which here takes the form of a deterministic equation for spatio-temporal evolution of the cumulative distribution function (CDF) of the random system state, as a means of forward uncertainty propagation. Uncertainty reduction is achieved by recasting the standard loss function, i.e. discrepancy between observations and model predictions, in distributional terms. This step exploits the equivalence between minimization of the square error discrepancy and the Kullback-Leibler divergence. The loss function is regularized by adding a Lagrangian constraint enforcing fulfilment of the CDF equation. Minimization is performed sequentially, progressively updating the parameters of the CDF equation as more measurements are assimilated.

MSC:

62B11 Information geometry (statistical aspects)
35L60 First-order nonlinear hyperbolic equations
62M20 Inference from stochastic processes and prediction
62M40 Random fields; image analysis
62R40 Topological data analysis

References:

[1] Tartakovsky DM, Gremaud PA. 2015 Method of distributions for uncertainty quantification. In Handbook of uncertainty quantification (eds R Ghanem, D Higdon, H Owhadi), pp. 763-783. New York, NY: Springer.
[2] Boso F, Broyda SV, Tartakovsky DM. 2014 Cumulative distribution function solutions of advection-reaction equations with uncertain parameters. Proc. R. Soc. A 470, 20140189. (doi:10.1098/rspa.2014.0189) · doi:10.1098/rspa.2014.0189
[3] Boso F, Tartakovsky DM. 2016 The method of distributions for dispersive transport in porous media with uncertain hydraulic properties. Water Resour. Res. 52, 4700-4712. (doi:10.1002/2016WR018745) · doi:10.1002/2016WR018745
[4] Alawadhi AA, Boso F, Tartakovsky DM. 2018 Method of distributions for water-hammer equations with uncertain parameters. Water Resour. Res. 54, 9398-9411. (doi:10.1029/2018WR023383) · doi:10.1029/2018WR023383
[5] Ghanem R, Red-Horse J. 2015 Polynomial chaos: modeling, estimation, and approximation. In Handbook of uncertainty quantification (eds R Ghanem, D Higdon, H Owhadi), pp. 1-31. New York, NY: Springer.
[6] Venturi D, Tartakovsky DM, Tartakovsky AM, Karniadakis GE. 2013 Exact PDF equations and closure approximations for advective-reactive transport. J. Comput. Phys. 243, 323-343. (doi:10.1016/j.jcp.2013.03.001) · Zbl 1349.35068 · doi:10.1016/j.jcp.2013.03.001
[7] Yang HJ, Boso F, Tchelepi HA, Tartakovsky DM. 2019 Probabilistic forecast of single-phase flow in porous media with uncertain properties. Water Resour. Res. 55, 8631-8645. (doi:10.1029/2019WR026090) · doi:10.1029/2019WR026090
[8] Tartakovsky DM, Dentz M, Lichtner PC. 2009 Probability density functions for advective-reactive transport in porous media with uncertain reaction rates. Water Resour. Res. 45, W07414. (doi:10.1029/2008WR007383) · doi:10.1029/2008WR007383
[9] Wikle CK, Berliner LM. 2007 A Bayesian tutorial for data assimilation. Physica D 230, 1-16. (doi:10.1016/j.physd.2006.09.017) · Zbl 1113.62032 · doi:10.1016/j.physd.2006.09.017
[10] Evensen G. 2009 Data assimilation: the ensemble Kalman filter. New York, NY: Springer. · Zbl 1395.93534
[11] Myung IJ. 2003 Tutorial on maximum likelihood estimation. J. Math. Psychol. 47, 90-100. (doi:10.1016/S0022-2496(02)00028-7) · Zbl 1023.62112 · doi:10.1016/S0022-2496(02)00028-7
[12] Cousineau D, Helie S. 2013 Improving maximum likelihood estimation using prior probabilities: a tutorial on maximum a posteriori estimation and an examination of the Weibull distribution. Tutor. Quant. Methods Psychol. 9, 61-71. (doi:10.20982/tqmp.09.2.p061) · doi:10.20982/tqmp.09.2.p061
[13] Katzfuss M, Stroud JR, Wikle CK. 2016 Understanding the ensemble Kalman filter. Am. Stat. 70, 350-357. (doi:10.1080/00031305.2016.1141709) · Zbl 07665895 · doi:10.1080/00031305.2016.1141709
[14] Brooks S, Gelman A, Jones G, Meng XL (eds). 2011 Handbook of Markov chain Monte Carlo. Boca Raton, FL: CRC Press. · Zbl 1218.65001
[15] Speekenbrink M. 2016 A tutorial on particle filters. J. Math. Psychol. 73, 140-152. (doi:10.1016/j.jmp.2016.05.006) · Zbl 1396.91657 · doi:10.1016/j.jmp.2016.05.006
[16] Blei DM, Kucukelbir A, McAuliffe JD. 2017 Variational inference: a review for statisticians. J. Am. Stat. Assoc. 112, 859-877. (doi:10.1080/01621459.2017.1285773) · doi:10.1080/01621459.2017.1285773
[17] Herzog R, Kunisch K. 2010 Algorithms for PDE-constrained optimization. GAMM-Mitt. 33, 163-176. (doi:10.1002/gamm.201010013) · Zbl 1207.49034 · doi:10.1002/gamm.201010013
[18] Raissi M, Perdikaris P, Karniadakis GE. 2017 Machine learning of linear differential equations using Gaussian processes. J. Comput. Phys. 348, 683-693. (doi:10.1016/j.jcp.2017.07.050) · Zbl 1380.68339 · doi:10.1016/j.jcp.2017.07.050
[19] Zhu Y, Zabaras N, Koutsourelakis PS, Perdikaris P. 2019 Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data. J. Comput. Phys. 394, 56-81. (doi:10.1016/j.jcp.2019.05.024) · Zbl 1452.68172 · doi:10.1016/j.jcp.2019.05.024
[20] Zhang D, Lu L, Guo L, Karniadakis GE. 2019 Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems. J. Comput. Phys. 397, 108850. (doi:10.1016/j.jcp.2019.07.048) · Zbl 1454.65008 · doi:10.1016/j.jcp.2019.07.048
[21] Wang P, Tartakovsky DM. 2013 CDF solutions of Buckley-Leverett equation with uncertain parameters. Multiscale Model. Simul. 11, 118-133. (doi:10.1137/120865574) · Zbl 1267.76088 · doi:10.1137/120865574
[22] Boso F, Tartakovsky DM. 2020 Data-informed method of distributions for hyperbolic conservation laws. SIAM J. Sci. Comput. 42, A559-A583. (doi:10.1137/19M1260773) · Zbl 1436.35330 · doi:10.1137/19M1260773
[23] Perthame B. 2002 Kinetic formulation of conservation laws, vol. 21. London, UK: Oxford University Press. · Zbl 1030.35002
[24] Cafaro C, Mancini S. 2011 Quantifying the complexity of geodesic paths on curved statistical manifolds through information geometric entropies and Jacobi fields. Physica D 240, 607-618. (doi:10.1016/j.physd.2010.11.013) · Zbl 1217.37079 · doi:10.1016/j.physd.2010.11.013
[25] Wang P, Tartakovsky DM. 2012 Uncertainty quantification in kinematic wave models. J. Comput. Phys. 231, 7868-7880. (doi:10.1016/j.jcp.2012.07.030) · Zbl 1257.35124 · doi:10.1016/j.jcp.2012.07.030
[26] Giffin A, Caticha A. 2007 Updating probabilities with data and moments. AIP Conf. Proc. 954, 74-84. (doi:10.1063/1.2821302) · doi:10.1063/1.2821302
[27] Bellemare MG, Danihelka I, Dabney W, Mohamed S, Lakshminarayanan B, Hoyer S, Munos R. 2018 The Cramer distance as a solution to biased wasserstein gradients. (http://arxiv.org/abs/1705.10743)
[28] Pinsker MS. 1964 Information and information stability of random variables and processes. San Francisco, CA: Holden-Day. · Zbl 0125.09202
[29] Topsoe F. 2000 Some inequalities for information divergence and related measures of discrimination. IEEE Trans. Inf. Theory 46, 1602-1609. (doi:10.1109/18.850703) · Zbl 1003.94010 · doi:10.1109/18.850703
[30] Stein EM, Shakarchi R. 2011 Functional analysis: introduction to further topics in analysis, vol. 4. Princeton, NJ: Princeton University Press. · Zbl 1235.46001
[31] Raissi M, Perdikaris P, Karniadakis GE. 2019 Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686-707. (doi:10.1016/j.jcp.2018.10.045) · Zbl 1415.68175 · doi:10.1016/j.jcp.2018.10.045
[32] Abadi M et al. 2015 TensorFlow: large-scale machine learning on heterogeneous systems. Software available from tensorflow.org.
[33] Barron AR. 1993 Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inf. Theory 39, 930-945. (doi:10.1109/18.256500) · Zbl 0818.68126 · doi:10.1109/18.256500
[34] Bölcskei H, Grohs P, Kutyniok G, Petersen P. 2019 Optimal approximation with sparsely connected deep neural networks. SIAM J. Math. Data Sci. 1, 8-45. (doi:10.1137/18M118709X) · Zbl 1499.41029 · doi:10.1137/18M118709X
[35] Amari SI. 2016 Information geometry and its applications. New York, NY: Springer. · Zbl 1350.94001
[36] Kullback S. 1997 Information theory and statistics. New York, NY: Wiley. · Zbl 0897.62003
[37] Cafaro C, Alsing PM. 2020 Information geometry aspects of minimum entropy production paths from quantum mechanical evolutions. Phys. Rev. E 101, 022110. (doi:10.1103/PhysRevE.101.022110) · doi:10.1103/PhysRevE.101.022110
[38] Guyer JE, Wheeler D, Warren JA. 2009 FiPy: partial differential equations with Python. Comput. Sci. Eng. 11, 6-15. (doi:10.1109/MCSE.2009.52) · doi:10.1109/MCSE.2009.52
[39] Crestani E, Camporese M, Baú D, Salandin P. 2013 Ensemble Kalman filter versus ensemble smoother for assessing hydraulic conductivity via tracer test data assimilation. Hydrol. Earth Syst. Sci. 17, 1517-1531. (doi:10.5194/hess-17-1517-2013) · doi:10.5194/hess-17-1517-2013
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.