×

\(l_p\) regularization for ensemble Kalman inversion. (English) Zbl 1487.65062

Summary: Ensemble Kalman inversion (EKI) is a derivative-free optimization method that lies between the deterministic and probabilistic approaches for inverse problems. EKI iterates the Kalman update of ensemble-based Kalman filters, whose ensemble converges to a minimizer of an objective function. EKI regularizes ill-posed problems by restricting the ensemble to the linear span of the initial ensemble, or by iterating regularization with early stopping. Another regularization approach for EKI, Tikhonov EKI, penalizes the objective function using the \(l_2\) penalty term, preventing overfitting in the standard EKI. This paper proposes a strategy to implement \(l_p\), \(0<p\leq 1\), regularization for EKI to recover sparse structures in the solution. The strategy transforms an \(l_p\) problem into an \(l_2\) problem, which is then solved by Tikhonov EKI. The transformation is explicit, and thus the proposed approach has a computational cost comparable to Tikhonov EKI. We validate the proposed approach’s effectiveness and robustness through a suite of numerical experiments, including compressive sensing and subsurface flow inverse problems.

MSC:

65J20 Numerical solutions of ill-posed problems in abstract spaces; regularization
65C05 Monte Carlo methods
35Q93 PDEs in connection with control and optimization
49M41 PDE constrained optimization (numerical aspects)

References:

[1] J. L. Anderson, An ensemble adjustment Kalman filter for data assimilation, Mon. Weather Rev., 129 (2001), pp. 2884-2903.
[2] J. M. Bardsley, A. Solonen, H. Haario, and M. Laine, Randomize-then-optimize: A method for sampling from posterior distributions in nonlinear inverse problems, SIAM J. Sci. Comput., 36 (2014), pp. A1895-A1910, https://doi.org/10.1137/140964023. · Zbl 1303.65003
[3] M. Benning and M. Burger, Modern regularization methods for inverse problems, Acta Numer., 27 (2018), pp. 1-111. · Zbl 1431.65080
[4] S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, Cambridge, MA, 2004. · Zbl 1058.90049
[5] A. C. Reynolds, M. Zafari, and G. Li, Iterative forms of the ensemble Kalman filter, in Proceedings of the 10th European Conference on the Mathematics of Oil Recovery, European Association of Geoscientists & Engineers, 2006, cp-23-00025.
[6] E. J. Candès, M. B. Wakin, and S. P. Boyd, Enhancing sparsity by reweighted \(l_1\) minimization, J. Fourier Anal. Appl., 14 (2008), pp. 877-905. · Zbl 1176.94014
[7] J. Carrera and S. P. Neuman, Estimation of aquifer parameters under transient and steady state conditions: 3. Application to synthetic and field data, Water Resour. Res., 22 (1986), pp. 228-242.
[8] N. K. Chada, A. M. Stuart, and X. T. Tong, Tikhonov regularization within ensemble Kalman inversion, SIAM J. Numer. Anal., 58 (2020), pp. 1263-1294, https://doi.org/10.1137/19M1242331. · Zbl 1447.35337
[9] N. K. Chada and X. T. Tong, Convergence Acceleration of Ensemble Kalman Inversion in Nonlinear Settings, preprint, https://arxiv.org/abs/1911.02424, 2019.
[10] J. Duchi, E. Hazan, and Y. Singer, Adaptive subgradient methods for online learning and stochastic optimization, J. Mach. Learn. Res., 12 (2011), pp. 2121-2159. · Zbl 1280.68164
[11] M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing, Springer, New York, 2010. · Zbl 1211.94001
[12] G. Evensen, Data Assimilation: The Ensemble Kalman Filter, Springer, London, 2009. · Zbl 1395.93534
[13] R. Fletcher, Practical Methods of Optimization, 2nd ed., John Wiley & Sons, Chichester, 1987. · Zbl 0905.65002
[14] S. Foucart and H. Rauhut, A Mathematical Introduction to Compressive Sensing, Birkhäuser, Basel, 2013. · Zbl 1315.94002
[15] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, Cambridge, MA, 2016. · Zbl 1373.68009
[16] G. A. Gottwald and A. J. Majda, A mechanism for catastrophic filter divergence in data assimilation for sparse observation networks, Nonlin. Processes Geophys., 20 (2013), pp. 705-712.
[17] M. Hanke, A regularizing Levenberg-Marquardt scheme, with applications to inverse groundwater filtration problems, Inverse Problems, 13 (1997), pp. 79-95. · Zbl 0873.65057
[18] J. Harlim and A. J. Majda, Catastrophic filter divergence in filtering nonlinear dissipative systems, Comm. Math. Sci., 8 (2010), pp. 27-43. · Zbl 1190.93096
[19] M. Iglesias, K. J. H. Law, and A. Stuart, Ensemble Kalman methods for inverse problems, Inverse Problems, 29 (2013), 045001. · Zbl 1311.65064
[20] M. A. Iglesias, A regularizing iterative ensemble Kalman method for PDE-constrained inverse problems, Inverse Problems, 32 (2016), 025002. · Zbl 1334.65110
[21] D. Kelly, A. J. Majda, and X. T. Tong, Concrete ensemble Kalman filters with rigorous catastrophic filter divergence, Proc. Nat. Acad. Sci., 112 (2015), pp. 10589-10594, https://doi.org/10.1073/pnas.1511063112.
[22] N. B. Kovachki and A. M. Stuart, Ensemble Kalman inversion: A derivative-free technique for machine learning tasks, Inverse Problems, 35 (2019), 095005. · Zbl 1430.68266
[23] Y. Lee, A. Majda, and D. Qi, Preventing catastrophic filter divergence using adaptive additive inflation for baroclinic turbulence, Mon. Weather Rev., 145 (2017), pp. 669-682.
[24] G. Li and A. C. Reynolds, An iterative ensemble Kalman filter for data assimilation, SPE J., 14 (2009), pp. 496-505.
[25] J. M. S. Andersen and L. Vandnberghe, CVXOPT: A Python Software for Convex Optimization, https://cvxopt.org.
[26] D. Oliver, A. C. Reynolds, and N. Liu, Inverse Theory for Petroleum Reservoir Characterization and History Matching, 1st ed., Cambridge University Press, Cambridge, UK, 2008.
[27] B. T. Polyak and A. B. Juditsky, Acceleration of stochastic approximation by averaging, SIAM J. Control Optim., 30 (1992), pp. 838-855, https://doi.org/10.1137/0330046. · Zbl 0762.62022
[28] H. Schaeffer, Learning partial differential equations via data discovery and sparse optimization, Proc. A., 473 (2017), 20160446. · Zbl 1404.35397
[29] C. Schillings and A. M. Stuart, Analysis of the ensemble Kalman filter for inverse problems, SIAM J. Numer. Anal., 55 (2017), pp. 1264-1290, https://doi.org/10.1137/16M105959X. · Zbl 1366.65101
[30] T. Schneider, A. M. Stuart, and J.-L. Wu, Imposing Kalman Inversion for Sparse Learning of Dynamical Systems from Time-Averaged Data, preprint, https://arxiv.org/abs/2007.06175, 2020.
[31] X. T. Tong, A. J. Majda, and D. Kelly, Nonlinear stability of the ensemble Kalman filter with adaptive covariance inflation, Comm. Math. Sci., 14 (2016), pp. 1283-1313. · Zbl 1370.37009
[32] Z. Wang, J. M. Bardsley, A. Solonen, T. Cui, and Y. M. Marzouk, Bayesian inverse problems with \(l\_1\) priors: A randomize-then-optimize approach, SIAM J. Sci. Comput., 39 (2017), pp. S140-S166, https://doi.org/10.1137/16M1080938. · Zbl 1422.65028
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.