Experimental design for nonparametric correction of misspecified dynamical models. (English) Zbl 1403.62140
Summary: We consider a class of misspecified dynamical models where the governing term is only approximately known. Under the assumption that observations of the system’s evolution are accessible for various initial conditions, our goal is to infer a nonparametric correction to the misspecified driving term such as to faithfully represent the system dynamics and devise system evolution predictions for unobserved initial conditions. We model the unknown correction term as a Gaussian Process and analyze the problem of efficient experimental design to find an optimal correction term under constraints such as a limited experimental budget. We suggest a novel formulation for experimental design for this Gaussian process and show that approximately optimal (up to a constant factor) designs may be efficiently derived by utilizing results from the literature on submodular optimization. Our numerical experiments exemplify the effectiveness of these techniques.
MSC:
| 62K05 | Optimal statistical designs |
| 37M05 | Simulation of dynamical systems |
| 62G08 | Nonparametric regression and quantile regression |
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 60G15 | Gaussian processes |
Keywords:
model misspecification; dynamical systems; experimental design; submodularity; Gaussian processesReferences:
| [1] | J.-F. P. Abascal, S. R. Arridge, D. Atkinson, R. Horesh, L. Fabrizi, M. De Lucia, L. Horesh, R. H. Bayford, and D. S. Holder, Use of anisotropic modelling in electrical impedance tomography: description of method and preliminary assessment of utility in imaging brain function in the adult human head, Neuroimage, 43 (2008), pp. 258–268. |
| [2] | N. M. Alexandrov, R. M. Lewis, C. R. Gumbert, L. L. Green, and P. A. Newman, Approximation and model management in aerodynamic optimization with variable-fidelity models, J. Aircraft, 38 (2001), pp. 1093–1101. |
| [3] | M. A. Álvarez, L. Rosasco, N. D. Lawrence, et al., Kernels for vector-valued functions: A review, Found. Trends Mach. Learn., 4 (2012), pp. 195–266. · Zbl 1301.68212 |
| [4] | A. Antoulas, Approximation of Large-Scale Dynamical Systems, SIAM, Philadelphia, 2005. · Zbl 1112.93002 |
| [5] | N. Aronszajn, Theory of Reproducing Kernels, Trans. Amer. Math. Soc., 68 (1950), pp. 337–404. · Zbl 0037.20701 |
| [6] | H. Avron, K. L. Clarkson, and D. P. Woodruff, Faster Kernel Ridge Regression using Sketching and Preconditioning, arXiv:1611.03220, 2016. · Zbl 1379.65008 |
| [7] | H. Avron and V. Sindhwani, High-performance kernel machines with implicit distributed optimization and randomization, Technometrics, 58 (2016), pp. 341–349. |
| [8] | H. Avron and S. Toledo, Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix, J. ACM, 58 (2011). · Zbl 1327.68331 |
| [9] | A. Berlinet and C. Thomas-Agnan, Reproducing Kernel Hilbert Spaces in Probability and Statistics, Springer, New York, 2011. · Zbl 1145.62002 |
| [10] | J. M. Bernardo, Expected information as expected utility, Ann. Statist., (1979), pp. 686–690. · Zbl 0407.62002 |
| [11] | P. Blanco, S. Urquiza, and R. Feijóo, Assessing the influence of heart rate in local hemodynamics through coupled 3d-1d-0d models, Int. J. Numer. Methods Biomed. Eng., 26 (2010), pp. 890–903. · Zbl 1193.92027 |
| [12] | B. Blight and L. Ott, A Bayesian approach to model inadequacy for polynomial regression, Biometrika, 62 (1975), pp. 79–88. · Zbl 0308.62062 |
| [13] | G. E. Box, W. G. Hunter, and S. J. Hunter, Statistics for Experimenters, John Wiley & Sons, New York, 1978. · Zbl 0394.62003 |
| [14] | G. E. Box and G. C. Tiao, Bayesian Inference in Statistical Analysis, Vol. 40, John Wiley & Sons, New York, 2011. · Zbl 0271.62044 |
| [15] | G. Calinescu, C. Chekuri, M. Pál, and J. Vondrák, Maximizing a Submodular Set Function Subject to a Matroid Constraint, in Integer Programming and Combinatorial Optimization, Springer, New York, 2007, pp. 182–196. · Zbl 1136.90449 |
| [16] | K. Chaloner and I. Verdinelli, Bayesian experimental design: A review, Statist. Sci., 10 (1995), pp. 273–304. · Zbl 0955.62617 |
| [17] | D. R. Chase, L.-Y. Chen, and R. A. York, Modeling the capacitive nonlinearity in thin-film BST varactors, IEEE Trans. Microw. Theory Tech., 53 (2005), pp. 3215–3220. |
| [18] | S. Chaturantabut and D. C. Sorensen, Nonlinear model reduction via discrete empirical interpolation, SIAM J. Sci. Comput., 32 (2010), pp. 2737–2764. · Zbl 1217.65169 |
| [19] | A. R. Conn, K. Scheinberg, and L. N. Vicente, Introduction to Derivative-Free Optimization, SIAM, Philadelphia, 2009. · Zbl 1163.49001 |
| [20] | B. T. Cox, S. R. Arridge, K. P. Köstli, and P. C. Beard, Two-dimensional quantitative photoacoustic image reconstruction of absorption distributions in scattering media by use of a simple iterative method, Appl. Opt., 45 (2006), pp. 1866–1875. |
| [21] | E. De Vito, V. Umanità, and S. Villa, An extension of Mercer theorem to matrix-valued measurable kernels, Appl. Comput. Harmon. Anal., 34 (2013), pp. 339–351. · Zbl 1283.46018 |
| [22] | J. Donea, S. Giuliani, and J. Halleux, An Arbitrary Lagrangian–Eulerian Finite Element Method for Transient Dynamic Fluid-Structure Interactions, Comput. Methods Appl. Mech. Engrg., 33 (1982), pp. 689–723, . · Zbl 0508.73063 |
| [23] | H. Fatoorehchi, H. Abolghasemi, and R. Zarghami, Analytical approximate solutions for a general nonlinear resistor–nonlinear capacitor circuit model, Appl. Math. Model., 39 (2015), pp. 6021–6031. · Zbl 1443.94006 |
| [24] | V. V. Fedorov, Theory of Optimal Experiments, Elsevier, New York, 1972. |
| [25] | U. Feige, A threshold of ln n for approximating set cover, J. ACM, 45 (1998), pp. 634–652. · Zbl 1065.68573 |
| [26] | L. Formaggia, J. F. Gerbeau, F. Nobile, and A. Quarteroni, Numerical treatment of defective boundary conditions for the Navier–Stokes equations, SIAM J. Numer. Anal., 40 (2002), pp. 376–401, . · Zbl 1020.35070 |
| [27] | S. Fujishige, Submodular Functions and Optimization, Vol. 58, Elsevier, New York, 2005. · Zbl 1119.90044 |
| [28] | C. W. Gear, Numerical Initial Value Problems in Ordinary Differential Equations, Prentice Hall PTR, New York, 1971. · Zbl 1145.65316 |
| [29] | J. Goh, D. Bingham, J. P. Holloway, M. J. Grosskopf, C. C. Kuranz, and E. Rutter, Prediction and Computer Model Calibration using Outputs from Multi-Fidelity Simulators, preprint, arXiv:1208.2716, 2012. |
| [30] | H. Goldstein, Classical Mechanics, Pearson Education India, Delhi, 1965. |
| [31] | E. Haber, U. Ascher, D. Aruliah, and D. Oldenburg, Fast simulation of 3d electromagnetic problems using potentials, J. Comput. Phys., 163 (2000), pp. 150–171. · Zbl 1145.78323 |
| [32] | E. Haber, L. Horesh, and L. Tenorio, Numerical methods for experimental design of large-scale linear ill-posed inverse problems, Inverse Problems, 24 (2008), 055012. · Zbl 1153.65062 |
| [33] | N. Hao, L. Horesh, and M. Kilmer, Nuclear Norm Optimization and Its Application to Observation Model Specification, in Compressed Sensing & Sparse Filtering, Springer, New York, 2014, pp. 95–122. · Zbl 1352.65165 |
| [34] | T. Hastie, R. Tibshirani, J. Friedman, T. Hastie, J. Friedman, and R. Tibshirani, The Elements of Statistical Learning, Vol. 2, Springer, New York, 2009. · Zbl 1273.62005 |
| [35] | X. He and L.-S. Luo, Lattice Boltzmann model for the incompressible Navier–Stokes equation, J. Stat. Phys., 88 (1997), pp. 927–944. · Zbl 0939.82042 |
| [36] | H. Helms and J. Thomas, Truncation error of sampling-theorem expansions, Proc. IRE, 50 (1962), pp. 179–184. |
| [37] | L. Horesh, E. Haber, and L. Tenorio, Optimal experimental design for the large-scale nonlinear ill-posed problem of impedance imaging, in Large-Scale Inverse Problems and Quantification of Uncertainty, John Wiley & Sons, New York, 2010, pp. 273–290. · Zbl 1189.65073 |
| [38] | L. Horesh, L. Liberti, H. Avron, and D. Nahamoo, Globally Convergent System and Method for Automated Model Discovery, U.S. Patent Office 14/755,942, 2015. |
| [39] | D. Jagerman, Bounds for truncation error of the sampling expansion, SIAM J. Appl. Math., 14 (1966), pp. 714–723. · Zbl 0221.65200 |
| [40] | M. Kennedy and A. O’Hagan, Predicting the output from a complex computer code when fast approximations are available, Biometrika, 87 (2000), pp. 1–13. · Zbl 0974.62024 |
| [41] | M. C. Kennedy and A. O’Hagan, Bayesian calibration of computer models, J. Roy. Statist. Soc. Ser. B, 63 (2001), pp. 425–464. · Zbl 1007.62021 |
| [42] | A. Krause and D. Golovin, Submodular function maximization, in Tractability: Practical Approaches to Hard Problems, Cambridge University Press, Cambridge, 2014, pp. 71–109. |
| [43] | A. Krause and C. E. Guestrin, Near-optimal nonmyopic value of information in graphical models, in Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, 2012. · Zbl 1192.68645 |
| [44] | Y. A. Kutoyants, Identification of Dynamical Systems with Small Noise, Vol. 300, Springer, New York, 2012. · Zbl 0831.62058 |
| [45] | R. Lam, L. Horesh, H. Avron, and K. Willcox, An optimization framework for hybrid first principles data-driven modeling, in Proceedings of the Copper Mountain Conference on Iterative Methods, 2016. |
| [46] | R. Lam, K. Willcox, and D. H. Wolpert, Bayesian optimization with a finite budget: An approximate dynamic programming approach, in Proceedings of Advances in Neural Information Processing Systems, 2016, pp. 883–891. |
| [47] | S. J. Leary, A. Bhaskar, and A. J. Keane, A knowledge-based approach to response surface modelling in multifidelity optimization, J. Global Optim., 26 (2003), pp. 297–319. · Zbl 1116.90411 |
| [48] | C. Lieberman and K. Willcox, Goal-oriented inference: Approach, linear theory, and application to advection diffusion, SIAM Rev., 55 (2013), pp. 493–519. · Zbl 1273.35303 |
| [49] | A. I. March, Multidelity Methods for Multidisciplinary System Design, Ph.D. thesis, Massachusetts Institute of Technology, 2012. |
| [50] | J. A. Martinez and B. A. Mork, Transformer modeling for low-and mid-frequency transients—A review, IEEE Trans. Power Del., 20 (2005), pp. 1625–1632. |
| [51] | K. McGoff, S. Mukherjee, N. Pillai, et al., Statistical inference for dynamical systems: A review, Stat. Surv., 9 (2015), pp. 209–252. · Zbl 1327.62458 |
| [52] | C. A. Micchelli, Y. Xu, and H. Zhang, Universal kernels, J. Mach. Learn. Res., 7 (2006), pp. 2651–2667. · Zbl 1222.68266 |
| [53] | M. Minoux, Accelerated Greedy Algorithms for Maximizing Submodular Set Functions, in Optimization Techniques, Springer, New York, 1978, pp. 234–243. · Zbl 0372.90128 |
| [54] | P. M. Muller and W. L. Sjogren, Mascons: Lunar mass concentrations, Science, 161 (1968), pp. 680–684. |
| [55] | G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher, An analysis of approximations for maximizing submodular set functions, Math. Program., 14 (1978), pp. 265–294. · Zbl 0374.90045 |
| [56] | J. G. Oxley, Matroid Theory, Vol. 3, Oxford University Press, Oxford, 2006. |
| [57] | B. Peherstorfer and K. Willcox, Dynamic data-driven reduced-order models, Comput. Methods Appl. Mech. Engrg., 291 (2015), pp. 21 – 41, . · Zbl 1425.65205 |
| [58] | M. Poloczek, J. Wang, and P. I. Frazier, Multi-Information Source Optimization, preprint, arXiv:1603.00389, 2016. |
| [59] | F. Pukelsheim, Optimal Design of Experiments, Vol. 50, SIAM, Philadelphia, 1993. · Zbl 0834.62068 |
| [60] | A. Rahimi and B. Recht, Random features for large-scale kernel machines, in Procedings of Advances in Neural Information Processing Systems, 2007, pp. 1177–1184. |
| [61] | D. Rajnarayan, Trading Risk and Performance for Engineering Design Optimization using Multifidelity Analyses, Ph.D. thesis, Stanford University, CA, 2009. |
| [62] | C. E. Rasmussen, Gaussian Processes for Machine Learning, MIT Press, Cambridge, MA, 2006. · Zbl 1177.68165 |
| [63] | A. Rosencwaig and A. Gersho, Theory of the photoacoustic effect with solids, J. Appl. Phys., 47 (1976), pp. 64–69, . |
| [64] | A. Rossi, F. Marzari, and P. Farinella, Orbital evolution around irregular bodies, Earth Planets Space, 51 (1999), pp. 1173–1180. |
| [65] | Z. Rosson, F. Hall, and T. Vogel, Orbital behavior around a nonuniform celestial body, in J. Phys. 750, IOP Publishing, 2016, p. 012022. |
| [66] | C. R. Shalizi et al., Dynamics of Bayesian updating with dependent data and misspecified models, Electron. J. Stat., 3 (2009), pp. 1039–1074. · Zbl 1326.62017 |
| [67] | C. Soize, E. Capiez-Lernout, J.-F. Durand, C. Fernandez, and L. Gagliardini, Probabilistic model identification of uncertainties in computational models for dynamical systems and experimental validation, Comput. Method Appl. Mech. Engrg., 198 (2008), pp. 150–163. · Zbl 1194.74552 |
| [68] | L. Tenorio, C. Lucero, V. Ball, and L. Horesh, Experimental design in the context of Tikhonov regularized inverse problems, Statist. Model., 13 (2013), pp. 481–507. · Zbl 07257469 |
| [69] | L. Thomsen, Weak elastic anisotropy, Geophysics, 51 (1986), pp. 1954–1966. |
| [70] | J. Tinsley Oden, S. Prudhomme, A. Romkes, and P. T. Bauman, Multiscale modeling of physical phenomena: Adaptive control of models, SIAM J. Sci. Comput., 28 (2006), pp. 2359–2389. · Zbl 1126.74006 |
| [71] | A. Tizzard, L. Horesh, R. J. Yerworth, D. S. Holder, and R. Bayford, Generating accurate finite element meshes for the forward model of the human head in EIT, Physiol. Meas., 26 (2005), pp. S251–S261. |
| [72] | L. N. Trefethen, Approximation Theory and Approximation Practice, SIAM, Philadelphia, 2013. · Zbl 1264.41001 |
| [73] | A. Veneziani and C. Vergara, Flow Rate Defective Boundary Conditions in Haemodynamics Simulations, Internat. J. Numer. Methods Fluids, 2005. · Zbl 1134.76748 |
| [74] | J. Virieux and S. Operto, An overview of full-waveform inversion in exploration geophysics, Geophysics, 74 (2009), pp. WCC1–WCC26. |
| [75] | D. Wei, O. Okazaki, K. Harumi, E. Harasawa, and H. Hosaka, Comparative simulation of excitation and body surface electrocardiogram with isotropic and anisotropic computer heart models, IEEE Trans. Biomed. Eng., 42 (1995), pp. 343–357. |
| [76] | J. A. C. Weideman, Computation of the complex error function, SIAM J. Numer. Anal., 31 (1994), pp. 1497–1518. · Zbl 0832.65011 |
| [77] | C. K. Williams, Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond, in Learning in Graphical Models, Springer, New York, 1998, pp. 599–621. · Zbl 0921.62121 |
| [78] | P. Zunino, Multidimensional pharmacokinetic models applied to the design of drug-eluting stents, Cardiovascular Eng., 4 (2004), pp. 181–191. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
