Deep learning methods for limited data problems in X-ray tomography. (English) Zbl 07914367
Chen, Ke (ed.) et al., Handbook of mathematical models and algorithms in computer vision and imaging. Mathematical imaging and vision. Springer Reference. Cham: Springer. 1183-1202 (2023).
Summary: Successful medical diagnosis heavily relies on the reconstruction and analysis of images showing organs, bones, and other structures in the interior of the human body. In the last couple of years, the stored image data has increased tremendously, and also the computing power of modern GPUs experienced huge progress. Machine learning methods, and in particular deep learning methods, are on the rise to tackle advanced image reconstruction and image analysis tasks to support medical doctors in their diagnostic routines. In this chapter, we focus on the reconstruction task; especially consider tomographic imaging problems with incomplete, corrupted, or noisy data; and demonstrate how deep learning methods enable us to solve such tasks in a unified manner. We present the basic ideas of these methods assuming paired training data (supervised learning) and utilizing only feed-forward networks. In particular, we illustrate the underlying concepts for missing data problems in classical computed tomography (CT), noting that most of the concepts can be transferred to other inverse imaging problems.
For the entire collection see [Zbl 1527.94003].
For the entire collection see [Zbl 1527.94003].
MSC:
| 92C55 | Biomedical imaging and signal processing |
| 68U10 | Computing methodologies for image processing |
| 68T07 | Artificial neural networks and deep learning |
References:
| [1] | Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. Inverse Probl. 10(6), 1217 (1994) · Zbl 0809.35151 |
| [2] | Adler, J., Öktem, O.: Solving ill-posed inverse problems using iterative deep neural networks. Inverse Probl. 33(12), 124007 (2017) · Zbl 1394.92070 |
| [3] | Adler, J., Öktem O.: Learned primal-dual reconstruction. IEEE Trans. Med. Imaging 37(6), 1322-1332 (2018) |
| [4] | Anirudh, R., Kim, H., Thiagarajan, J.J., Mohan, K.A., Champley, K., Bremer, T.: Lose the views: limited angle CT reconstruction via implicit sinogram completion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6343-6352 (2018) |
| [5] | Arridge, S., Maass, P., Öktem, O., Schönlieb, C.-B.: Solving inverse problems using data-driven models. Acta Numer. 28, 1-174 (2019) · Zbl 1429.65116 |
| [6] | Bayaraa, T., Hyun, C.M., Jang, T.J., Lee, S.M., Seo, J.K.: A two-stage approach for beam hardening artifact reduction in low-dose dental CBCT. IEEE Access 8, 225981-225994 (2020) |
| [7] | Beard, P.: Biomedical photoacoustic imaging. Interface Focus 1(4), 602-631 (2011) |
| [8] | Boink, Y.E., Brune, C.: Learned SVD: solving inverse problems via hybrid autoencoding. arXiv preprint arXiv:1912.10840 (2019) |
| [9] | Boink, Y.E., Manohar, S., Brune, C.: A partially-learned algorithm for joint photo-acoustic reconstruction and segmentation. IEEE Trans. Medi. Imaging 39(1), 129-139 (2019) |
| [10] | Boink, Y.E., Haltmeier, M., Holman, S., Schwab, J.: Data-consistent neural networks for solving nonlinear inverse problems. arXiv preprint arXiv:2003.11253 (2020) |
| [11] | Bubba, T.A., Kutyniok, G., Lassas, M., Maerz, M., Samek, W., Siltanen, S., Srinivasan, V.: Learning the invisible: a hybrid deep learning-shearlet framework for limited angle computed tomography. Inverse Probl. 35(6), 064002 (2019) · Zbl 1416.92099 |
| [12] | Chen, H., Zhang, Y., Kalra, M.K., Lin, F., Chen, Y., Liao, P., Zhou, J., Wang, G.: Low-dose CT with a residual encoder-decoder convolutional neural network. IEEE Trans. Med. Imaging 36(12), 2524-2535 (2017) |
| [13] | Chen, G., Hong, X., Ding, Q., Zhang, Y., Chen, H., Fu, S., Zhao, Y., Zhang, X., Ji, H., Wang, G. et al.: Airnet: fused analytical and iterative reconstruction with deep neural network regularization for sparse-data CT. Med. Phys. 47(7), 2916-2930 (2020) |
| [14] | Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. J. Issued Courant Inst. Math. Sci. 57(11), 1413-1457 (2004) · Zbl 1077.65055 |
| [15] | Deans, S.R.: The Radon Transform and Some of Its Applications. Courier Corporation. Dover Publications, INC., Mineola, New York (2007) · Zbl 1121.44004 |
| [16] | Elad, M.: Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer Science & Business Media, New York (2010) · Zbl 1211.94001 |
| [17] | Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems, vol. 375. Springer Science & Business Media, Dordrecht (1996) · Zbl 0859.65054 |
| [18] | Frikel, J., Quinto, E.T.: Characterization and reduction of artifacts in limited angle tomography. Inverse Probl. 29(12), 125007 (2013) · Zbl 1284.92044 |
| [19] | Ghani, M.U., Karl, W.C.: CNN based sinogram denoising for low-dose CT. In: Mathematics in Imaging, pp. MM2D-5. Optical Society of America, Optical Society of America, Orlando, Florida (2018) |
| [20] | Ghani, M.U., Karl, W.C.: Fast enhanced CT metal artifact reduction using data domain deep learning. IEEE Trans. Comput. Imaging, IEEE Trans. Comput. Imaging, vol. 6, 181-193 (2019) |
| [21] | Gjesteby, L., Shan, H., Yang, Q., Xi, Y., Claus, B., Jin, Y., De Man, B., Wang, G.: Deep neural network for CT metal artifact reduction with a perceptual loss function. In: Proceedings of the Fifth International Conference on Image Formation in X-Ray Computed Tomography, vol. 1 (2018) |
| [22] | Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge, MA (2016) · Zbl 1373.68009 |
| [23] | Gu, J., Ye, J.C.: Multi-scale wavelet domain residual learning for limited-angle CT reconstruction. arXiv preprint arXiv:1703.01382 (2017) |
| [24] | Guazzo, A.: Deep learning for PET imaging: from denoising to learned primal-dual reconstruction (2020) |
| [25] | Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science & Business Media (2009) · Zbl 1273.62005 |
| [26] | Hauptmann, A., Adler, J., Arridge, S.R., Oktem, O.: Multi-scale learned iterative reconstruction. IEEE Trans. Comput. Imaging, vol. 6, 843-856 (2020) |
| [27] | Hecht-Nielsen, R.: Theory of the backpropagation neural network. In: Neural Networks for Perception, pp. 65-93. Elsevier (1992) |
| [28] | Higham, C.F., Higham, D.J.: Deep learning: an introduction for applied mathematicians. SIAM Rev. 61(4), 860-891 (2019) · Zbl 1440.68214 |
| [29] | Hounsfield, G.N.: Computerized transverse axial scanning (tomography): part 1. description of system. Br. J. Radiol. 46(552), 1016-1022 (1973) |
| [30] | Huang, Y., Huang, X., Taubmann, O., Xia, Y., Haase, V., Hornegger, J., Lauritsch, G., Maier, A.: Restoration of missing data in limited angle tomography based on Helgason-Ludwig consistency conditions. Biomed. Phys. Eng. Express 3(3), 035015 (2017) |
| [31] | Huang, X., Wang, J., Tang, F., Zhong, T., Zhang, Y.: Metal artifact reduction on cervical CT images by deep residual learning. Biomed. Eng. Online 17(1), 175 (2018a) |
| [32] | Huang, Y., Würfl, T., Breininger, K., Liu, L., Lauritsch, G., Maier, A.: Some investigations on robustness of deep learning in limited angle tomography. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 145-153. Springer (2018b) |
| [33] | Kang, E., Min, J., Ye, J.C.: A deep convolutional neural network using directional wavelets for low-dose x-ray CT reconstruction. Med. Phys. 44(10), e360-e375 (2017) |
| [34] | Kuanar, S., Athitsos, V., Mahapatra, D., Rao, K.R., Akhtar, Z., Dasgupta, D.: Low dose abdominal CT image reconstruction: an unsupervised learning based approach. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1351-1355. IEEE (2019) |
| [35] | Kwon, T., Ye, J.C.: Cycle-free cyclegan using invertible generator for unsupervised low-dose CT denoising. arXiv preprint arXiv:2104.08538 (2021) |
| [36] | Landweber, L.: An iteration formula for Fredholm integral equations of the first kind. Am. J. Math. 73(3), 615-624 (1951) · Zbl 0043.10602 |
| [37] | LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436-444 (2015) |
| [38] | Lee, J., Gu, J., Ye, J.C.: Unsupervised CT metal artifact learning using attention-guided beta-cyclegan. arXiv preprint arXiv:2007.03480 (2020) |
| [39] | Leuschner, J., Schmidt, M., Baguer, D.O., Maass, P.: LoDoPab-CT, a benchmark dataset for low-dose computed tomography reconstruction. Sci. Data 8(1), 1-12 (2021) |
| [40] | Li, H., Schwab, J., Antholzer, S., Haltmeier, M.: Nett: solving inverse problems with deep neural networks. Inverse Probl. 36(6), 065005 (2020) · Zbl 1456.65038 |
| [41] | Lundervold, A.S., Lundervold, A.: An overview of deep learning in medical imaging focusing on mri. Zeitschrift für Medizinische Physik 29(2), 102-127 (2019) |
| [42] | Lunz, S., Öktem, O., Schönlieb, C.-B.: Adversarial regularizers in inverse problems. arXiv preprint arXiv:1805.11572 (2018) |
| [43] | Mukherjee, S., Dittmer, S., Shumaylov, Z., Lunz, S., Öktem, O., Schönlieb, C.-B.: Learned convex regularizers for inverse problems. arXiv preprint arXiv:2008.02839 (2020) |
| [44] | Natterer, F.: The Mathematics of Computerized Tomography. SIAM, Philadelphia (2001) · Zbl 0973.92020 |
| [45] | Obmann, D., Nguyen, L., Schwab, J., Haltmeier, M.: Sparse anett for solving inverse problems with deep learning. In: 2020 IEEE 17th International Symposium on Biomedical Imaging Workshops (ISBI Workshops), pp. 1-4. IEEE (2020) |
| [46] | Park, H.S., Chung, Y.E., Lee, S.M., Kim, H.P., Seo, J.K.: Sinogram-consistency learning in CT for metal artifact reduction. arXiv preprint arXiv:1708.00607, 1 (2017) |
| [47] | Purcell, E.M., Torrey, H.C., Pound, R.V.: Resonance absorption by nuclear magnetic moments in a solid. Phys. Rev. 69(1-2), 37 (1946) |
| [48] | Quinto, E.T.: Tomographic reconstructions from incomplete data-numerical inversion of the exterior radon transform. Inverse Probl. 4(3), 867 (1988) · Zbl 0663.65136 |
| [49] | Ruder, S.: An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609. 04747 (2016) |
| [50] | Scherzer, O., Grasmair, M., Grossauer, H., Haltmeier, M., Lenzen, F.: Variational Methods in Imaging. Springer, New York (2009) · Zbl 1177.68245 |
| [51] | Schwab, J., Antholzer, S., Nuster, R., Haltmeier, M.: Real-time photoacoustic projection imaging using deep learning. arXiv preprint arXiv:1801.06693 (2018) |
| [52] | Schwab, J., Antholzer, S., Haltmeier, M.: Big in Japan: regularizing networks for solving inverse problems. J. Math. Imaging Vis., vol. 62, 445-455 (2019a) · Zbl 1434.68525 |
| [53] | Schwab, J., Antholzer, S., Haltmeier, M.: Deep null space learning for inverse problems: convergence analysis and rates. Inverse Probl. 35(2), 025008 (2019b) · Zbl 1491.65039 |
| [54] | Schwab, J., Antholzer, S., Haltmeier, M.: Learned backprojection for sparse and limited view photoacoustic tomography. In: Photons Plus Ultrasound: Imaging and Sensing 2019, vol. 10878, p. 1087837. International Society for Optics and Photonics, SPIE BiOS, San Francisco, California (2019c) |
| [55] | Shan, H., Padole, A., Homayounieh, F., Kruger, U., Khera, R.D., Nitiwarangkul, C., Kalra, M.K., Wang, G.: Competitive performance of a modularized deep neural network compared to commercial algorithms for low-dose CT image reconstruction. Nat. Mach. Intell. 1(6), 269-276 (2019) |
| [56] | Wang, G.: A perspective on deep imaging. IEEE Access 4, 8914-8924 (2016) |
| [57] | Werbos, P.: Beyond regression: new tools for prediction and analysis in the behavioral sciences. Ph. D. dissertation, Harvard University (1974) |
| [58] | Wu, D., Kim, K., El Fakhri, G., Li, Q.: Iterative low-dose CT reconstruction with priors trained by artificial neural network. IEEE Trans. Med. Imaging 36(12), 2479-2486 (2017) |
| [59] | Wu, D., Kim, K., Kalra, M.K., De Man, B., Li, Q.: Learned primal-dual reconstruction for dual energy computed tomography with reduced dose. In: 15th International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine, vol. 11072, p. 1107206. International Society for Optics and Photonics (2019) |
| [60] | Würfl, T., Hoffmann, M., Christlein, V., Breininger, K., Huang, Y., Unberath, M., Maier, A.K.: Deep learning computed tomography: learning projection-domain weights from image domain in limited angle problems. IEEE Trans. Med. Imaging 37(6), 1454-1463 (2018) |
| [61] | Zhang, Y., Yu, H.: Convolutional neural network based metal artifact reduction in x-ray computed tomography. IEEE Trans. Med. Imaging 37(6), 1370-1381 (2018) |
| [62] | Zhu, B., Liu, J.Z., Cauley, S.F., Rosen, B.R., Rosen, M.S.: Image reconstruction by domain-transform manifold learning. Nature 555(7697), 487-492 (2018) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
