Abstract
Inverse problems arise in a variety of imaging applications, including computed tomography, non-destructive testing, and remote sensing. Characteristic features of inverse problems are the non-uniqueness and instability of their solutions. Therefore, any reasonable solution method requires the use of regularization tools that select specific solutions and, at the same time, stabilize the inversion process. Recently, data-driven methods using deep learning techniques and neural networks showed to significantly outperform classical solution methods for inverse problems. In this chapter, we give an overview of inverse problems and demonstrate the necessity of regularization concepts for their solution. We show that neural networks can be used for the data-driven solution of inverse problems and review existing deep learning methods for inverse problems. In particular, we view these deep learning methods from the perspective of regularization theory, the mathematical foundation of stable solution methods for inverse problems. This chapter is more than just a review as many of the presented theoretical results extend existing ones.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
A is positive if: y ≥ 0 ⇒Ay ≥ 0.
References
Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. Inverse Probl. 10(6), 1217–1229 (1994)
Adler, J., Öktem, O.: Solving ill-posed inverse problems using iterative deep neural networks. Inverse Probl. 33(12), 124007 (2017)
Aggarwal, H.K., Mani, M.P., Jacob, M.: MoDL: model-based deep learning architecture for inverse problems. IEEE Trans. Med. Imaging 38(2), 394–405 (2018)
Agostinelli, F., Hoffman, M., Sadowski, P., Baldi, P.: Learning activation functions to improve deep neural networks. arXiv:1412.6830 (2014)
Aljadaany, R., Pal, D.K., Savvides, M.: Douglas-rachford networks: learning both the image prior and data fidelity terms for blind image deconvolution. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 10235–10244 (2019)
Arridge, S., Maass, P., Öktem, O., Schönlieb C.: Solving inverse problems using data-driven models. Acta Numer. 28, 1–174 (2019)
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. 35(8), 1798–1828 (2013)
Boink, Y.E., Haltmeier, M., Holman, S., Schwab, J.: Data-consistent neural networks for solving nonlinear inverse problems. arXiv:2003.11253 (2020), to apper in Inverse Probl. Imaging
Bora, A., Jalal, A., Price, E., Dimakis, A.G.: Compressed sensing using generative models. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 537–546 (2017)
Brosch, T., Tam, R., et al.: Manifold learning of brain MRIs by deep learning. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 633–640. Springer (2013)
Bubba, T.A., Kutyniok, G., Lassas, M., Maerz, M., Samek, W., Siltanen, S., Srinivasan, V.: Learning the invisible: a hybrid deep learning-shearlet framework for limited angle computed tomography. Inverse Probl. 35(6), 064002 (2019)
Chen, D., Davies, M.E.: Deep decomposition learning for inverse imaging problems. In European Conference on Computer Vision, pp. 510–526). Springer, Cham (2020)
Chen, T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D.K.: Neural ordinary differential equations. In: Advances in Neural Information Processing Systems, pp. 6571–6583 (2018)
Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57(11), 1413–1457 (2004)
Dittmer, S., Maass, P.: A projectional ansatz to reconstruction. arXiv:1907.04675 (2019)
Dittmer, S., Kluth, T., Maass, P., Baguer, D.O.: Regularization by architecture: a deep prior approach for inverse problems. J. Math. Imaging Vis. 62, 456–470 (2020)
Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems, vol. 375. Kluwer Academic Publishers Group, Dordrecht (1996)
Georg, M., Souvenir, R., Hope, A., Pless, R.: Manifold learning for 4D CT reconstruction of the lung. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of 13th International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, London (2016)
Grasmair, M., Haltmeier, M., Scherzer, O.: Sparse regularization with lq penalty term. Inverse Probl. 24(5), 055020 (2008)
Han, Y., Yoo, J.J., Ye, J.C.: Deep residual learning for compressed sensing CT reconstruction via persistent homology analysis (2016). http://arxiv.org/abs/1611.06391
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, Y., Preuhs, A., Manhart, M., Lauritsch, G., Maier, A.: Data consistent ct reconstruction from insufficient data with learned prior images. arXiv:2005.10034 (2020)
Ivanov, V.K., Vasin, V.V., Tanana, V.P.: Theory of Linear Ill-Posed Problems and Its Applications. Inverse and Ill-Posed Problems Series, 2nd edn. VSP, Utrecht, (2002). Translated and revised from the 1978 Russian original
Jin, K.H., McCann, M.T., Froustey, E., Unser, M.: Deep convolutional neural network for inverse problems in imaging. IEEE Trans. Image Process. 26(9), 4509–4522 (2017)
Kobler, E., Klatzer, T., Hammernik, K., Pock, T.: Variational networks: connecting variational methods and deep learning. In: German Conference on Pattern Recognition, pp. 281–293. Springer (2017)
Kofler, A., Haltmeier, M., Kolbitsch, C., Kachelrieß, M., Dewey, M.: A U-Nets cascade for sparse view computed tomography. In: Proceedings of 1st Workshop on Machine Learning for Medical Image Reconstruction, pp. 91–99. Springer (2018)
Kofler, A., Haltmeier, M., Schaeffter, T., Kachelrieß, M., Dewey, M., Wald, C., Kolbitsch, C.: Neural networks-based regularization of large-scale inverse problems in medical imaging. Phys. Med. Biol. 65, 135003 (2020)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Li, H., Schwab, J., Antholzer, S., Haltmeier, M.: NETT: solving inverse problems with deep neural networks. Inverse Probl. 36, 065005 (2020)
Lindenstrauss, J., Tzafriri, L.: On the complemented subspaces problem. Israel J. Math. 9(2), 263–269 (1971)
Lunz, S., Öktem, O., Schönlieb, C.: Adversarial regularizers in inverse problems. In: Advances in Neural Information Processing Systems, vol. 31, pp. 8507–8516 (2018)
Mardani, M., Gong, E., Cheng, J.Y., Vasanawala, S.S., Zaharchuk, G., Xing, L., Pauly, J.M.: Deep generative adversarial neural networks for compressive sensing MRI. IEEE Trans. Med. Imag. 38(1), 167–179 (2018)
Nashed, M.Z.: Inner, outer, and generalized inverses in banach and hilbert spaces. Numer. Func. Anal. Opt. 9(3–4), 261–325 (1987)
Obmann, D., Nguyen, L., Schwab, J., Haltmeier, M.: Sparse aNETT for solving inverse problems with deep learning. In 2020 IEEE 17th International Symposium on Biomedical Imaging Workshops (ISBI Workshops) (pp. 1–4). IEEE (2020a)
Obmann, D., Schwab, J., Haltmeier, M.: Deep synthesis network for regularizing inverse problems. Inverse Problems, 37(1), 015005 (2020b)
Obmann, D., Nguyen, L., Schwab, J., Haltmeier, M.: Augmented NETT regularization of inverse problems. J. Phys. Commun. 5(10), 105002 (2021)
Phillips, R.S.: On linear transformations. Trans. Am. Math. Soc. 48(3), 516–541 (1940)
Ramachandran, P., Zoph, B., Le, Q.V.: Searching for activation functions. arXiv:1710.05941 (2017)
Resmerita, E., Anderssen, R.S.: Joint additive Kullback–Leibler residual minimization and regularization for linear inverse problems. Math. Methods Appl. Sci. 30(13), 1527–1544 (2007)
Roth, S., Black, M.J.: Fields of experts: a framework for learning image priors. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 860–867. IEEE (2005)
Scherzer, O., Grasmair, M., Grossauer, H., Haltmeier, M., Lenzen, F.: Variational Methods in Imaging. Applied Mathematical Sciences, vol. 167. Springer, New York (2009)
Schlemper, J., Caballero, J., Hajnal, J.V., Price, A., Rueckert, D.: A deep cascade of convolutional neural networks for MR image reconstruction. In: Proceedings of Information Processing in Medical Imaging, pp. 647–658. Springer (2017)
Schwab, J., Antholzer, S., Haltmeier, M.: Deep null-space learning for inverse problems: convergence analysis and rates. Inverse Probl. 35(2), 025008 (2019)
Schwab, J., Antholzer, S., Haltmeier, M.: Big in Japan: regularizing networks for solving inverse problems. J. Math. Imaging Vis. 62, 445–455 (2020)
Sulam, J., Aberdam, A., Beck, A., Elad, M.: On multi-layer basis pursuit, efficient algorithms and convolutional neural networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(8), 1968–1980 (2019)
Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 9446–9454 (2018)
Van Veen, D., Jalal, A., Soltanolkotabi, M., Price, E., Vishwanath, S., Dimakis, A.G.: Compressed sensing with deep image prior and learned regularization. arXiv:1806.06438 (2018)
Wachinger, C., Yigitsoy, M., Rijkhorst, E., Navab, N.: Manifold learning for image-based breathing gating in ultrasound and MRI. Med. Image Anal. 16(4), 806–818 (2012)
Yang, Y., Sun, J., Li, H., Xu, Z.: Deep ADMM-net for compressive sensing MRI. In: Proceedings of 30th International Conference on Neural Information Processing Systems, pp. 10–18 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this entry
Cite this entry
Haltmeier, M., Nguyen, L. (2023). Regularization of Inverse Problems by Neural Networks. In: Chen, K., Schönlieb, CB., Tai, XC., Younes, L. (eds) Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging. Springer, Cham. https://doi.org/10.1007/978-3-030-98661-2_81
Download citation
DOI: https://doi.org/10.1007/978-3-030-98661-2_81
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98660-5
Online ISBN: 978-3-030-98661-2
eBook Packages: Mathematics and StatisticsReference Module Computer Science and Engineering