On learned operator correction in inverse problems. (English) Zbl 1474.65182
Summary: We discuss the possibility of learning a data-driven explicit model correction for inverse problems and whether such a model correction can be used within a variational framework to obtain regularized reconstructions. This paper discusses the conceptual difficulty of learning such a forward model correction and proceeds to present a possible solution as a forward-adjoint correction that explicitly corrects in both data and solution spaces. We then derive conditions under which solutions to the variational problem with a learned correction converge to solutions obtained with the correct operator. The proposed approach is evaluated on an application to limited view photoacoustic tomography and compared to the established framework of the Bayesian approximation error method.
MSC:
| 65K10 | Numerical optimization and variational techniques |
| 65F22 | Ill-posedness and regularization problems in numerical linear algebra |
| 94A08 | Image processing (compression, reconstruction, etc.) in information and communication theory |
| 47A52 | Linear operators and ill-posed problems, regularization |
| 68T07 | Artificial neural networks and deep learning |
Keywords:
model correction; inverse problems; operator learning; deep learning; variational methods; photoacoustic tomographyReferences:
| [1] | J. Adler and O. Öktem, Solving ill-posed inverse problems using iterative deep neural networks, Inverse Problems, 33 (2017), 124007. · Zbl 1394.92070 |
| [2] | J. Adler and O. Öktem, Learned primal-dual reconstruction, IEEE Trans. Med. Imaging, 37 (2018), pp. 1322-1332. |
| [3] | S. Antholzer, M. Haltmeier, and J. Schwab, Deep learning for photoacoustic tomography from sparse data, Inverse Prob. Sci. Eng., 27 (2019), pp. 987-1005. · Zbl 1465.94008 |
| [4] | S. Arridge, J. Kaipio, V. Kolehmainen, M. Schweiger, E. Somersalo, T. Tarvainen, and M. Vauhkonen, Approximation errors and model reduction with an application in optical diffusion tomography, Inverse Problems, 22 (2006), pp. 175-195. · Zbl 1138.65042 |
| [5] | S. Arridge, P. Maass, O. Öktem, and C.-B. Schönlieb, Solving inverse problems using data-driven models, Acta Numer., 28 (2019), pp. 1-174. · Zbl 1429.65116 |
| [6] | P. Beard, Biomedical photoacoustic imaging, Interface Focus, 1 (2011), pp. 602-631. |
| [7] | Y. E. Boink and C. Brune, Learned SVD: Solving Inverse Problems via Hybrid Autoencoding, preprint, https://arxiv.org/abs/1912.10840 (2019). |
| [8] | L. Borcea, V. Druskin, A. Mamanov, S. Moskow, and M. Zaslvsky, Reduced order models for spectral domain inversion: Embedding into the continuous problem and generation of internal data, Inverse Problems, 36 (2020), 055010. · Zbl 1478.65106 |
| [9] | T. A. Bubba, G. Kutyniok, M. Lassas, M. Maerz, W. Samek, S. Siltanen, and V. Srinivasan, Learning the invisible: A hybrid deep learning-shearlet framework for limited angle computed tomography, Inverse Problems, 35 (2019), 064002. · Zbl 1416.92099 |
| [10] | M. Burger, Y. Korolev, and J. Rasch, Convergence rates and structure of solutions of inverse problems with imperfect forward models, Inverse Problems, 35 (2019), 024006. · Zbl 1408.49033 |
| [11] | B. Cox and P. Beard, Fast calculation of pulsed photoacoustic fields in fluids using k-space methods, J. Acoust. Soc. Amer., 117 (2005), pp. 3616-3627. |
| [12] | N. Davoudi, X. L. Deán-Ben, and D. Razansky, Deep learning optoacoustic tomography with sparse data, Nature Mach. Intell., 1 (2019), pp. 453-460. |
| [13] | H. Egger, J.-F. Pietschmann, and M. Schlottbom, Identification of chemotaxis models with volume-filling, SIAM J. Appl. Math., 75 (2015), pp. 275-288. · Zbl 1328.35305 |
| [14] | R. W. Freund, Model reduction methods based on Krylov subspaces, Acta Numer., 12 (2003), pp. 267-319. · Zbl 1046.65021 |
| [15] | S. Guan, A. Khan, S. Sikdar, and P. Chitnis, Fully dense UNet for \(2\) D sparse photoacoustic tomography artifact removal, IEEE J. Biomed. Health Inform., 24 (2020), pp. 568-576. |
| [16] | S. J. Hamilton and A. Hauptmann, Deep D-bar: Real-time electrical impedance tomography imaging with deep neural networks, IEEE Trans. Med. Imaging, 37 (2018), pp. 2367-2377. |
| [17] | K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock, and F. Knoll, Learning a variational network for reconstruction of accelerated MRI data, Magn. Reson. Med., 79 (2018), pp. 3055-3071. |
| [18] | A. Hauptmann, J. Adler, S. Arridge, and O. Öktem, Multi-scale learned iterative reconstruction, IEEE Trans. Comput. Imaging, to appear. |
| [19] | A. Hauptmann, S. Arridge, F. Lucka, V. Muthurangu, and J. A. Steeden, Real-time cardiovascular MR with spatio-temporal artifact suppression using deep learning-proof of concept in congenital heart disease, Magn. Reson. Med., 81 (2019), pp. 1143-1156. |
| [20] | A. Hauptmann, B. Cox, F. Lucka, N. Huynh, M. Betcke, P. Beard, and S. Arridge, Approximate k-space models and deep learning for fast photoacoustic reconstruction, in International Workshop on Machine Learning for Medical Image Reconstruction, Springer, Cham, Switzerland, 2018, pp. 103-111. |
| [21] | A. Hauptmann, F. Lucka, M. Betcke, N. Huynh, J. Adler, B. Cox, P. Beard, S. Ourselin, and S. Arridge, Model based learning for accelerated, limited-view 3d photoacoustic tomography, IEEE Trans. Med. Imaging, 39 (2018), pp. 1382-1393. |
| [22] | K. H. Jin, M. T. McCann, E. Froustey, and M. Unser, Deep convolutional neural network for inverse problems in imaging, IEEE Trans. Image Process., 26 (2017), pp. 4509-4522. · Zbl 1409.94275 |
| [23] | J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Appl. Math. Sci. 160, Springer, New York, 2005. · Zbl 1068.65022 |
| [24] | J. Kaipio and E. Somersalo, Statistical inverse problems: Discretization, model reduction and inverse crimes, J. Comput. Appl. Math., 198 (2007), pp. 493-504. · Zbl 1101.65008 |
| [25] | E. Kang, J. Min, and J. C. Ye, A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction, Med. Phys., 44 (2017), pp. e360-e375. |
| [26] | M. C. Kennedy and A. O’Hagan, Bayesian calibration of computer models, J. R. Stat. Soc. Ser. B Stat. Methodol., 63 (2001), pp. 425-464. · Zbl 1007.62021 |
| [27] | E. Kobler, A. Effland, K. Kunisch, and T. Pock, Total deep variation for linear inverse problems, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Piscataway, NJ, 2020, pp. 7549-7558. |
| [28] | K. P. Koestli, M. Frenz, H. Bebie, and H. P. Weber, Temporal backward projection of optoacoustic pressure transients using Fourier transform methods, Phys. Med. Biol., 46 (2001), pp. 1863-1872. |
| [29] | Y. Korolev and J. Lellmann, Image reconstruction with imperfect forward models and applications in deblurring, SIAM J. Imaging Sci., 11 (2018), pp. 197-218. · Zbl 1401.94018 |
| [30] | P. Kuchment and L. Kunyansky, Mathematics of thermoacoustic tomography, European J. Appl. Math., 19 (2008), pp. 191-224. · Zbl 1185.35327 |
| [31] | H. Li, J. Schwab, S. Antholzer, and M. Haltmeier, Nett: Solving inverse problems with deep neural networks, Inverse Problems, 36 (2020), 065005. · Zbl 1456.65038 |
| [32] | A. Lorz, J.-F. Pietschmann, and M. Schlottbom, Parameter identification in a structured population model, Inverse problems, 35 (2019), 095008. · Zbl 1422.92122 |
| [33] | S. Lunz, O. Öktem, and C.-B. Schönlieb, Adversarial regularizers in inverse problems, in Advances in Neural Information Processing Systems, Curran Associates, Red Hook, NY, 2018, pp. 8507-8516. |
| [34] | O. Ronneberger, P. Fischer, and T. Brox, U-net: Convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Cham, Switzerland, 2015, pp. 234-241. |
| [35] | T. Sahlström, A. Pulkkinen, J. Tick, J. Leskinen, and T. Tarvainen, Modeling of errors due to uncertainties in ultrasound sensor locations in photoacoustic tomography, IEEE Trans. Med. Imaging, 39 (2020), pp. 2140-2150. |
| [36] | J. Schlemper, J. Caballero, J. V. Hajnal, A. N. Price, and D. Rueckert, A deep cascade of convolutional neural networks for dynamic MR image reconstruction, IEEE Trans. Med. Imaging, 37 (2017), pp. 491-503. |
| [37] | J. Schwab, S. Antholzer, and M. Haltmeier, Deep null space learning for inverse problems: Convergence analysis and rates, Inverse Problems, 35 (2019), 025008. · Zbl 1491.65039 |
| [38] | J. H. Siewerdsen and D. A. Jaffray, Cone-beam computed tomography with a flat-panel imager: Magnitude and effects of X-ray scatter, Med. Phys., 28 (2001), pp. 220-231. |
| [39] | D. Smyl and D. Liu, Less is often more: Applied inverse problems using hp-forward models, J. Comput. Phys., 399 (2019), 108949. · Zbl 1453.74080 |
| [40] | D. Smyl, T. N. Tallman, J. A. Black, A. Hauptmann, and D. Liu, Learning and Correcting Non-Gaussian Model Errors, preprint, https://arxiv.org/abs/2005.14592 (2020). |
| [41] | T. Tarvainen, A. Pulkkinen, B. T. Cox, J. P. Kaipio, and S. R. Arridge, Bayesian image reconstruction in quantitative photoacoustic tomography, IEEE Trans. Med. Imaging, 32 (2013), pp. 2287-2298. |
| [42] | B. E. Treeby and B. T. Cox, k-wave: Matlab toolbox for the simulation and reconstruction of photoacoustic wave fields, J. Biomed. Opt., 15 (2010), 021314. |
| [43] | B. E. Treeby, J. Jaros, A. P. Rendell, and B. T. Cox, Modeling nonlinear ultrasound propagation in heterogeneous media with power law absorption using a k-space pseudospectral method., J. Acoust. Soc. Amer., 131 (2012), pp. 4324-4336, https://doi.org/10.1121/1.4712021. |
| [44] | V. Vishnevskiy, R. Rau, and O. Goksel, Deep variational networks with exponential weighting for learning computed tomography, in International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Cham, Switzerland, 2019, pp. 310-318. |
| [45] | L. W. Y Xu, G. Ambartsoumian, and P. Kuchment, Reconstructions in limited-view thermoacoustic tomography, Med. Phys., 31 (2004), pp. 724-733. |
| [46] | E. Zhang, J. Laufer, and P. Beard, Backward-mode multiwavelength photoacoustic scanner using a planar Fabry-Perot polymer film ultrasound sensor for high-resolution three-dimensional imaging of biological tissues, Appl. Opt., 47 (2008), pp. 561-577. |
| [47] | L. Zhu, Y. Xie, J. Wang, and L. Xing, Scatter correction for cone-beam CT in radiation therapy, Med. Phys., 36 (2009), pp. 2258-2268. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
