Hierarchical ensemble Kalman methods with sparsity-promoting generalized gamma hyperpriors. (English) Zbl 07805180
Summary: This paper introduces a computational framework to incorporate flexible regularization techniques in ensemble Kalman methods, generalizing the iterative alternating scheme to nonlinear inverse problems. The proposed methodology approximates the maximum a posteriori (MAP) estimate of a hierarchical Bayesian model characterized by a conditionally Gaussian prior and generalized gamma hyperpriors. Suitable choices of hyperparameters yield sparsity-promoting regularization. We propose an iterative algorithm for MAP estimation, which alternates between updating the unknown with an ensemble Kalman method and updating the hyperparameters in the regularization to promote sparsity. The effectiveness of our methodology is demonstrated in several computed examples, including compressed sensing and subsurface flow inverse problems.
MSC:
| 62F15 | Bayesian inference |
| 35Q62 | PDEs in connection with statistics |
| 65C20 | Probabilistic models, generic numerical methods in probability and statistics |
Keywords:
ensemble Kalman methods; sparsity-promoting regularization; iterative alternating scheme; hierarchical inverse problemsReferences:
| [1] | S. H. D. A. Agrawal Kim Sanz-Alonso Strang, A variational inference approach to inverse problems with gamma hyperpriors, SIAM/ASA Journal on Uncertainty Quantification, 10, 1533-1559 (2022) · Zbl 1514.35425 · doi:10.1137/21M146209X |
| [2] | D. M. G. Armbruster Herty Visconti, A stabilization of a continuous limit of the ensemble Kalman inversion, SIAM Journal on Numerical Analysis, 60, 1494-1515 (2022) · Zbl 1501.37090 · doi:10.1137/21M1414000 |
| [3] | S. P. O. C.-B. Arridge Maass Öktem Schönlieb, Solving inverse problems using data-driven models, Acta Numerica, 28, 1-174 (2019) · Zbl 1429.65116 · doi:10.1017/S0962492919000059 |
| [4] | J. F. Bard, Practical Bilevel Optimization: Algorithms and Applications, volume 30., Springer Science & Business Media, 2013. |
| [5] | B. M. F. W. Bell Cathey, The iterated Kalman filter update as a Gauss-Newton method, IEEE Transactions on Automatic Control, 38, 294-297 (1993) · Zbl 0775.93237 · doi:10.1109/9.250476 |
| [6] | D. C. P. Blömker Schillings Wacker, A strongly convergent numerical scheme from ensemble Kalman inversion, SIAM Journal on Numerical Analysis, 56, 2537-2562 (2018) · Zbl 1432.60067 · doi:10.1137/17M1132367 |
| [7] | D. C. P. S. Blömker Schillings Wacker Weissmann, Well posedness and convergence analysis of the ensemble Kalman inversion, Inverse Problems, 35, 085007 (2019) · Zbl 1432.62325 · doi:10.1088/1361-6420/ab149c |
| [8] | D. A. F. E. B. Calvetti Pascarella Pitolli Somersalo Vantaggi, A hierarchical Krylov-Bayes iterative inverse solver for MEG with physiological preconditioning, Inverse Problems, 31, 125005 (2015) · Zbl 1329.35346 · doi:10.1088/0266-5611/31/12/125005 |
| [9] | D. A. F. E. B. Calvetti Pascarella Pitolli Somersalo Vantaggi, Brain activity mapping from MEG data via a hierarchical Bayesian algorithm with automatic depth weighting, Brain Topography, 32, 363-393 (2019) · doi:10.1007/s10548-018-0670-7 |
| [10] | D. F. E. B. Calvetti Pitolli Somersalo Vantaggi, Bayes meets Krylov: Statistically inspired preconditioners for CGLS, SIAM Review, 60, 429-461 (2018) · Zbl 1392.65047 · doi:10.1137/15M1055061 |
| [11] | D. M. E. Calvetti Pragliola Somersalo, Sparsity promoting hybrid solvers for hierarchical Bayesian inverse problems, SIAM Journal on Scientific Computing, 42, A3761-A3784 (2020) · Zbl 07303422 · doi:10.1137/20M1326246 |
| [12] | D. M. E. A. Calvetti Pragliola Somersalo Strang, Sparse reconstructions from few noisy data: Analysis of hierarchical Bayesian models with generalized gamma hyperpriors, Inverse Problems, 36, 025010 (2020) · Zbl 1464.62365 · doi:10.1088/1361-6420/ab4d92 |
| [13] | D. E. A. Calvetti Somersalo Strang, Hierachical Bayesian models and sparsity: \(L^2\)-magic, Inverse Problems, 35, 035003 (2019) · Zbl 1490.62078 · doi:10.1088/1361-6420/aaf5ab |
| [14] | C. M. Carvalho, N. G. Polson and J. G. Scott, Handling sparsity via the horseshoe, In Artificial Intelligence and Statistics, 73-80. PMLR, 2009. |
| [15] | N. K. Y. D. Chada Chen Sanz-Alonso, Iterative ensemble Kalman methods: A unified perspective with some new variants, Foundations of Data Science, 3, 331-369 (2021) · Zbl 1493.65180 · doi:10.3934/fods.2021011 |
| [16] | N. K. M. A. L. A. M. Chada Iglesias Roininen Stuart, Parameterizations for ensemble Kalman inversion, Inverse Problems, 34, 055009 (2018) · Zbl 1515.62041 · doi:10.1088/1361-6420/aab6d9 |
| [17] | N. K. A. M. X. T. Chada Stuart Tong, Tikhonov regularization within ensemble Kalman inversion, SIAM Journal on Numerical Analysis, 58, 1263-1294 (2020) · Zbl 1447.35337 · doi:10.1137/19M1242331 |
| [18] | N. K. X. Chada Tong, Convergence acceleration of ensemble Kalman inversion in nonlinear settings, Mathematics of Computation, 91, 1247-1280 (2022) · Zbl 1486.49049 · doi:10.1090/mcom/3709 |
| [19] | Y. D. R. Chen Sanz-Alonso Willett, Autodifferentiable ensemble Kalman filters, SIAM Journal on Mathematics of Data Science, 4, 801-833 (2022) · Zbl 1493.62499 · doi:10.1137/21M1434477 |
| [20] | P. D. Congdon, Applied Bayesian Hierarchical Methods, Chapman and Hall/CRC, 2010. · Zbl 1216.62036 |
| [21] | I. R. M. C. S. Daubechies DeVore Fornasier Güntürk, Iteratively reweighted least squares minimization for sparse recovery, Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences, 63, 1-38 (2010) · Zbl 1202.65046 · doi:10.1002/cpa.20303 |
| [22] | Z. Q. Ding Li, Ensemble Kalman sampler: Mean-field limit and convergence analysis, SIAM Journal on Mathematical Analysis, 53, 1546-1578 (2021) · Zbl 1468.35205 · doi:10.1137/20M1339507 |
| [23] | G. Evans, Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics, Journal of Geophysical Research: Oceans, 99, 10143-10162 (1995) · doi:10.1029/94JC00572 |
| [24] | G. P. J. Evans van Leeuwen, Assimilation of Geosat altimeter data for the Agulhas current using the ensemble Kalman filter with a quasigeostrophic model, Monthly Weather Review, 124, 85-96 (1996) · doi:10.1175/1520-0493(1996)124<0085:AOGADF>2.0.CO;2 |
| [25] | G. Evensen, Data Assimilation: The Ensemble Kalman Filter, volume 2., Springer, 2009. · Zbl 1395.93534 |
| [26] | A. Gelman, J. B. Carlin, H. S. Stern and D. B. Rubin, Bayesian Data Analysis, Chapman and Hall/CRC, 1995. |
| [27] | O. A. Ghattas and D. Sanz-Alonso, Non-Asymptotic analysis of ensemble Kalman updates: Effective dimension and localization, arXiv preprint, arXiv: 2208.03246, 2022. |
| [28] | I. F. B. D. Gorodnitsky Rao, Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm, IEEE Transactions on Signal Processing, 45, 600-616 (1997) · doi:10.1109/78.558475 |
| [29] | P. J. Green, Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives, Journal of the Royal Statistical Society: Series B (Methodological), 46, 149-170 (1984) · Zbl 0555.62028 · doi:10.1111/j.2517-6161.1984.tb01288.x |
| [30] | C. W. Groetsch, Inverse Problems in the Mathematical Sciences, volume 52., Springer, 1993. · Zbl 0779.45001 |
| [31] | Y. D. S. Gu Oliver, An iterative ensemble Kalman filter for multiphase fluid flow data assimilation, SPE Journal, 12, 438-446 (2007) · doi:10.2118/108438-PA |
| [32] | M. Hanke, A regularizing Levenberg-Marquardt scheme, with applications to inverse groundwater filtration problems, Inverse Problems, 13, 79-95 (1997) · Zbl 0873.65057 · doi:10.1088/0266-5611/13/1/007 |
| [33] | M. G. Herty Visconti, Kinetic methods for inverse problems, Kinetic & Related Models, 12, 1109-1130 (2019) · Zbl 1420.35417 · doi:10.3934/krm.2019042 |
| [34] | D. Z. Huang, J. Huang, S. Reich and A. M. Stuart, Efficient derivative-free Bayesian inference for large-scale inverse problems, Inverse Problems, 38 (2022), Paper No. 125006, 40 pp, arXiv preprint, arXiv: 2204.04386, 2022. · Zbl 07611872 |
| [35] | M. A. Iglesias, A regularizing iterative ensemble Kalman method for PDE-constrained inverse problems, Inverse Problems, 32, 025002 (2016) · Zbl 1334.65110 · doi:10.1088/0266-5611/32/2/025002 |
| [36] | M. A. K. J. H. A. M. Iglesias Law Stuart, Ensemble Kalman methods for inverse problems, Inverse Problems, 29, 045001 (2014) · Zbl 1311.65064 · doi:10.1088/0266-5611/29/4/045001 |
| [37] | M. A. Y. Iglesias Yang, Adaptive regularisation for ensemble Kalman inversion, Inverse Problems, 37, 025008 (2021) · Zbl 1456.65095 · doi:10.1088/1361-6420/abd29b |
| [38] | S. I. Kabanikhin, Definitions and examples of inverse and ill-posed problems, J. Inverse Ill-Posed Probl., 16, 317-357 (2008) · Zbl 1170.35100 · doi:10.1515/JIIP.2008.019 |
| [39] | V. Y. A. I. Korolev Zeifman, Generalized negative binomial distributions as mixed geometric laws and related limit theorems, Lithuanian Mathematical Journal, 59, 366-388 (2019) · Zbl 1448.60044 · doi:10.1007/s10986-019-09452-x |
| [40] | N. B. A. M. Kovachki Stuart, Ensemble Kalman inversion: A derivative-free technique for machine learning tasks, Inverse Problems, 35, 095005 (2019) · Zbl 1430.68266 · doi:10.1088/1361-6420/ab1c3a |
| [41] | K. J. H. V. Law Zankin, Sparse online variational Bayesian regression, SIAM/ASA Journal on Uncertainty Quantification, 10, 1070-1100 (2022) · Zbl 1498.62019 · doi:10.1137/21M1401188 |
| [42] | Y. Lee, \( \ell_p\) regularization for ensemble Kalman inversion, SIAM Journal on Scientific Computing, 43, A3417-A3437 (2021) · Zbl 1487.65062 · doi:10.1137/20M1365168 |
| [43] | G. Li and A. C. Reynolds, An iterative ensemble Kalman filter for data assimilation, In SPE Annual Technical Conference and Exhibition, Society of Petroleum Engineers, 2007. |
| [44] | C. Louizos, M. Welling and D. P. Kingma, Learning sparse neural networks through \(\ell_0\) regularization, arXiv preprint, arXiv: 1712.01312, 2017. |
| [45] | V. A. Morozov, On the solution of functional equations by the method of regularization, In Doklady Akademii Nauk, volume 167,510-512. Russian Academy of Sciences, 1966. · Zbl 0187.12203 |
| [46] | B. K. Natarajan, Sparse approximate solutions to linear systems, SIAM Journal on Computing, 24, 227-234 (1995) · Zbl 0827.68054 · doi:10.1137/S0097539792240406 |
| [47] | J. Pan, Z. Hu, Z. Su and M.-H. Yang, Deblurring text images via \(\ell_0\)-regularized intensity and gradient prior, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, 2901-2908. |
| [48] | J. Pidstrigach and S. Reich, Affine-invariant ensemble transform methods for logistic regression, Foundations of Computational Mathematics, 2022, 1-34. · Zbl 07673617 |
| [49] | A. C. Reynolds, M. Zafari and G. Li, Iterative forms of the ensemble Kalman filter, In ECMOR X-10th European Conference on the Mathematics of Oil Recovery, cp-23. European Association of Geoscientists & Engineers, 2006. |
| [50] | H. E. Robbins, An empirical Bayes approach to statistics, In Breakthroughs in Statistics, Springer, 1992,388-394. |
| [51] | D. Sanz-Alonso, A. M. Stuart and A. Taeb, Inverse problems and data assimilation, arXiv preprint, arXiv: 1810.06191, 2019. · Zbl 1535.62001 |
| [52] | C. A. M. Schillings Stuart, Analysis of the ensemble Kalman filter for inverse problems, SIAM Journal on Numerical Analysis, 55, 1264-1290 (2017) · Zbl 1366.65101 · doi:10.1137/16M105959X |
| [53] | T. S. A. J. Schneider Lan Stuart Teixeira, Earth system modeling 2.0: A blueprint for models that learn from observations and targeted high-resolution simulations, Geophysical Research Letters, 44, 12-396 (2017) · doi:10.1002/2017GL076101 |
| [54] | T. A. M. J.-L. Schneider Stuart Wu, Ensemble kalman inversion for sparse learning of dynamical systems from time-averaged data, Journal of Computational Physics, 470, 111559 (2022) · Zbl 1541.37097 · doi:10.1016/j.jcp.2022.111559 |
| [55] | A. M. Stuart, Inverse problems: A Bayesian perspective, Acta Numerica, 19, 451-559 (2010) · Zbl 1242.65142 · doi:10.1017/S0962492910000061 |
| [56] | R. Tibshirani, Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society: Series B (Methodological), 58, 267-288 (1996) · Zbl 0850.62538 · doi:10.1111/j.2517-6161.1996.tb02080.x |
| [57] | P. Tseng, Convergence of a block coordinate descent method for nondifferentiable minimization, Journal of Optimization Theory and Applications, 109, 475-494 (2001) · Zbl 1006.65062 · doi:10.1023/A:1017501703105 |
| [58] | S. Ungarala, On the iterated forms of Kalman filters using statistical linearization, Journal of Process Control, 22, 935-943 (2012) · doi:10.1016/j.jprocont.2012.03.004 |
| [59] | C. K. L. M. Wikle Berliner, A Bayesian tutorial for data assimilation, Physica D: Nonlinear Phenomena, 230, 1-16 (2007) · Zbl 1113.62032 · doi:10.1016/j.physd.2006.09.017 |
| [60] | H. Zou, The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 101, 1418-1429 (2006) · Zbl 1171.62326 · doi:10.1198/016214506000000735 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
