×

2D and 3D image reconstruction from slice data based on a constrained bilateral smoothing and dynamic mode decomposition. (English) Zbl 1510.94017

Summary: In this paper, we present an algebraic, graph theoretic and data-based image inpainting algorithm. The algorithm is applied to reconstruct area or volume from one-dimensional and two-dimensional slice data. Given one-dimensional or two-dimensional slice data, our algorithm first performs a simple algebraic presmoothing of the data, e.g., Gaussian filters. The second step is to obtain, from these smoothed data, an operator that reconstructs them via Dynamic Mode Decomposition. Dynamic Mode Decomposition is an equation-free modeler, which is designed to construct the discrete dynamical operator considering slice data as a time series data. We interpret the DMD as a solution to the inverse problem and apply an appropriate continuous extension of DMD to fill the missing time series data. The last step is to smooth the outcome from the second step using a Constraint Bilateral Smoother. The Constraint Bilateral Smoother is designed to smooth the boundary of area or volume, respecting the given slice data. We demonstrate the performance of the proposed algorithm, in several test cases, which include an Magnetic Resonance Imaging of a two-year-old and a Computed Tomography scan of a Covid-19 patient. We also show that a couple of test results are better than those from some competitive alternative techniques up to 50%. Our tests also show that the inpainting computation time increases linearly as a function of the number of pixels or voxels with slope just around 0.5.

MSC:

94A08 Image processing (compression, reconstruction, etc.) in information and communication theory
68U10 Computing methodologies for image processing
92C55 Biomedical imaging and signal processing
Full Text: DOI

References:

[1] Mri, 2021. https://www.mayoclinic.org/tests-procedures/mri/about.
[2] D. W. Mcrobbie, E. A. Moore, M. J. Graves, MRI from Picture to Proton10.1017/9781107706958.020
[3] Smith-Bindman, R.; Miglioretti, D. L.; Larson, E. B., Rising use of diagnostic medical imaging in a large integrated health system, Health Aff., 27, 6, 1491-1502 (2008)
[4] Kovačević, N.; Henderson, J. T.; Chan, E.; Lifshitz, N.; Bishop, J.; Evans, A. C.; Henkelman, R. M.; Chen, X. J., A three-dimensional MRI atlas of the mouse brain with estimates of the average and variability, Cereb. Cortex, 15, 5, 639-645 (2005)
[5] Kim, J.; Lee, C.-O., Three-dimensional volume reconstruction using two-dimensional parallel slices, SIAM J. Imaging Sci., 12, 1-27 (2019) · Zbl 1423.94007
[6] Kim, S.-U.; Lee, C.-O., Accurate surface reconstruction in 3D using two-dimensional parallel cross sections, J. Math. Imaging Vis., 53, 2, 182-195 (2015) · Zbl 1343.94011
[7] Rudin, L. I.; Osher, S., Total variation based image restoration with free local constraints, Proceedings of 1st International Conference on Image Processing, vol. 1, 31-35 (1994), IEEE
[8] Li, Y.; Shin, J.; Choi, Y.; Kim, J., Three-dimensional volume reconstruction from slice data using phase-field models, Comput. Vis. Image Underst., 137, 115-124 (2015)
[9] Burger, M.; He, L.; Schönlieb, C.-B., Cahn-Hilliard inpainting and a generalization for grayvalue images, SIAM J. Imaging Sci., 2, 4, 1129-1167 (2009) · Zbl 1180.49007
[10] Kutz, J. N.; Brunton, S. L.; Brunton, B. W.; Proctor, J. L., Dynamic Mode Decomposition: Data-driven Modeling of Complex Systems (2016), SIAM · Zbl 1365.65009
[11] Azencot, O.; Yin, W.; Bertozzi, A., Consistent dynamic mode decomposition, SIAM J. Appl. Dyn. Syst., 18, 3, 1565-1585 (2019) · Zbl 1428.37098
[12] SCHMID, P. J., Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656, 5-28 (2010) · Zbl 1197.76091
[13] Schmid, P. J.; Meyer, K. E.; Pust, O., Dynamic mode decomposition and proper orthogonal decomposition of flow in a lid-driven cylindrical cavity, 8th International Symposium on Particle Image Velocimetry, 25-28 (2009)
[14] Shi, J.; Malik, J., Normalized cuts and image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 22, 8, 888-905 (2000)
[15] Ojeda-Ruiz, I.; Lee, Y.-J., A fast constrained image segmentation algorithm, Results Appl. Math., 100103 (2020) · Zbl 1452.68255
[16] G. Jo, D. Kang, Y.-J. Lee, I. Ojeda-Ruiz, 3D normalized cut problems with intervening contours, In preparation (2021).
[17] Tomasi, C.; Manduchi, R., Bilateral filtering for gray and color images, Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271), 839-846 (1998), IEEE
[18] http://papers.nips.cc/paper/1830-learning-segmentation-by-random-walks.pdf
[19] Knyazev, A., Edge-enhancing filters with negative weights, 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), 260-264 (2015), IEEE
[20] Ku, J.; Lee, Y. J.; Sheen, D., A hybrid two-step finite element method for flux approximation: a priori estimates, ESAIM: Math. Model. Numer.Anal., 51, 4, 1303-1316 (2017) · Zbl 1379.65089
[21] Schmid, P. J., Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656, 5-28 (2010) · Zbl 1197.76091
[22] Stewart, G. W., On the early history of the singular value decomposition, SIAM Rev., 35, 4, 551-566 (1993) · Zbl 0799.01016
[23] S. Kim, M. Kim, S. Lee, Y.J. Lee, Discovering spatial-temporal patterns of COVID-19 pandemic in South Korea, To appearSci. Rep.(2021).
[24] J.H. Tu, C.W. Rowley, D.M. Luchtenburg, S.L. Brunton, J.N. Kutz, On dynamic mode decomposition: theory and applications, arXiv preprint arXiv:1312.0041(2013). · Zbl 1346.37064
[25] Fernández-Martínez, J. L.; Pallero, J. L.G.; Fernández-Muñiz, Z.; Pedruelo-González, L. M., The effect of noise and Tikhonov’s regularization in inverse problems. Part I: the linear case, J. Appl. Geophys., 108, 176-185 (2014)
[26] O’Meara, K. C.; Vinsonhaler, C., On approximately simultaneously diagonalizable matrices, Linear Algebra Appl., 412, 1, 39-74 (2006) · Zbl 1087.15507
[27] Wang, R., Singular vector perturbation under gaussian noise, SIAM J. Matrix Anal. Appl., 36, 1, 158-177 (2015) · Zbl 1315.15011
[28] Vu, V., Singular vectors under random perturbation, Random Struct. Algorithms, 39, 4, 526-538 (2011) · Zbl 1242.65069
[29] Hetzel, A. J.; Liew, J. S.; Morrison, K. E., The probability that a matrix of integers is diagonalizable, Am. Math. Mon., 114, 6, 491-499 (2007) · Zbl 1140.15018
[30] I.S. Gousias, Pediatric brain atlases, 2008. http://brain-development.org/brain-atlases/pediatric-brain-atlases/.
[31] O. Paiva, Coronacases.org - coronacases_org_003.nii, 2020. https://www.kaggle.com/andrewmvd/covid19-ct-scans?select=ct_scans.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.