Image segmentation with adaptive spatial priors from joint registration. (English) Zbl 1538.65042
Summary: Image segmentation is a crucial but challenging task that has many applications. In medical imaging, for instance, intensity inhomogeneity and noise are common. In thigh muscle images, different muscles are closely packed together and there are often no clear boundaries between them. Intensity based segmentation models cannot separate one muscle from another. To solve such problems, in this work we present a segmentation model with adaptive spatial priors from joint registration. This model combines segmentation and registration in a unified framework to leverage their positive mutual influence. The segmentation is based on a modified Gaussian mixture model, which integrates intensity inhomogeneity and spatial smoothness. The registration plays the role of providing a shape prior. We adopt a modified sum of squared difference fidelity term and Tikhonov regularity term for registration and also utilize a Gaussian pyramid and parametric method for robustness. The connection between segmentation and registration is guaranteed by the cross entropy metric that aims to make the segmentation map (from segmentation) and deformed atlas (from registration) as similar as possible. This joint framework is implemented within a constraint optimization framework, which leads to an efficient algorithm. We evaluate our proposed model on synthetic and thigh muscle MR images. Numerical results show the improvement as compared to segmentation and registration performed separately and other joint models.
MSC:
| 65D18 | Numerical aspects of computer graphics, image analysis, and computational geometry |
| 35R30 | Inverse problems for PDEs |
| 65K10 | Numerical optimization and variational techniques |
| 92C55 | Biomedical imaging and signal processing |
Keywords:
image segmentation; shape priors; image registration; intensity inhomogeneity; joint model; Gaussian mixture model; variational method; thigh muscle segmentationSoftware:
GitHubReferences:
| [1] | E. Ahmad, M. H. Yap, H. Degens, and J. S. McPhee, Atlas-registration based image segmentation of MRI human thigh muscles in 3D space, in Proceedings of Medical Imaging 2014: Image Perception, Observer Performance, and Technology Assessment, SPIE, 2014, pp. 424-435. |
| [2] | S. Andrews and G. Hamarneh, The generalized log-ratio transformation: Learning shape and adjacency priors for simultaneous thigh muscle segmentation, IEEE Trans. Med. Imaging, 34 (2015), pp. 1773-1787. |
| [3] | J. Ashburner and K. J. Friston, Unified segmentation, Neuroimage, 26 (2005), pp. 839-851. |
| [4] | T. Atta-Fosu and W. Guo, Joint segmentation and nonlinear registration using fast Fourier transform and total variation, in Research in Shape Analysis, Assoc. Women Math. Ser. 12, A. Genctav, K. Leonard, S. Tari, E. Hubert, G. Morin, N. El-Zehiry, and E. Chambers, eds., Springer, New York, 2018, pp. 111-132. · Zbl 1421.94004 |
| [5] | P. Y. Baudin, N. Azzabou, P. G. Carlier, and N. Paragios, Prior knowledge, random walks and human skeletal muscle segmentation, in Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, New York, 2012, pp. 569-576. |
| [6] | C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, Oxford, 1995. · Zbl 0868.68096 |
| [7] | C. L. Chan, C. Anitescu, Y. Zhang, and T. Rabczuk, Two and three dimensional image registration based on B-spline composition and level sets, Commun. Comput. Phys., 21 (2017), pp. 600-622. · Zbl 1475.68422 |
| [8] | Y. Chen, W. Guo, F. Huang, D. Wilson, and E. A. Geiser, Using prior shape and points in medical image segmentation, in Proceedings of the International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition, Springer, New York, 2003, pp. 291-305. |
| [9] | N. Debroux, J. Aston, F. Bonardi, A. Forbes, C. L. Guyader, M. Romanchikova, and C. B. Schonlieb, A variational model dedicated to joint segmentation, registration, and atlas generation for shape analysis, SIAM J. Imaging Sci., 13 (2020), pp. 351-380. · Zbl 07196107 |
| [10] | N. Debroux and C. L. Guyader, A joint segmentation/registration model based on a nonlocal characterization of weighted total variation and nonlocal shape descriptors, SIAM J. Imaging Sci., 11 (2018), pp. 957-990. · Zbl 1437.94014 |
| [11] | S. Esedo-Lu and F. Otto, Threshold dynamics for networks with arbitrary surface tensions, Comm. Pure Appl. Math., 68 (2015), pp. 808-864. · Zbl 1334.82072 |
| [12] | S. Ghosh, N. Ray, and P. Boulanger, A structured deep-learning based approach for the automated segmentation of human leg muscle from 3D MRI, in Proceedings of the 14th Conference on Computer and Robot Vision, IEEE, 2017, pp. 117-123. |
| [13] | A. Gooya, K. M. Pohl, M. Bilello, L. Cirillo, G. Biros, E. R. Melhem, and C. Davatzikos, GLISTR: Glioma image segmentation and registration, IEEE Trans. Med. Imaging, 31 (2012), pp. 1941-1954. |
| [14] | Z. Guo, H. Zhang, Z. Chen, E. van der Plas, L. Gutmann, D. Thedens, P. Nopoulos, and M. Sonka, Fully automated 3D segmentation of MR-imaged calf muscle compartments: Neighborhood relationship enhanced fully convolutional network, Comput. Med. Imaging Graph., 87 (2021), 101835. |
| [15] | C. L. Guyader and L. A. Vese, A combined segmentation and registration framework with a nonlinear elasticity smoother, Comput. Vis. Image Underst., 115 (2011), pp. 1689-1709. |
| [16] | E. Jolivet, E. Dion, P. Rouch, G. Dubois, R. Charrier, C. Payan, and W. Skalli, Skeletal muscle segmentation from MRI dataset using a model-based approach, Comput. Methods Biomech. Biomed. Eng. Imaging Vis., 2 (2014), pp. 138-145. |
| [17] | J. Kemnitz, C. F. Baumgartner, F. Eckstein, A. Chaudhari, A. Ruhdorfer, W. Wirth, S. K. Eder, and E. Konukoglu, Clinical evaluation of fully automated thigh muscle and adipose tissue segmentation using a U-Net deep learning architecture in context of osteoarthritic knee pain, Magn. Reson. Mater. Phy., 33 (2020), pp. 483-493. |
| [18] | J. Kemnitz, F. Eckstein, A. G. Culvenor, A. Ruhdorfer, T. Dannhauer, S. Ring-Dimitriou, A. M. Sänger, and W. Wirth, Validation of an active shape model-based semi-automated segmentation algorithm for the analysis of thigh muscle and adipose tissue cross-sectional areas, Magn. Reson. Mater. Phy., 30 (2017), pp. 489-503. |
| [19] | J. Liu, X. Wang, and X.-C. Tai, Deep convolutional neural networks with spatial regularization, volume and star-shape priors for image segmentation, J. Math. Imaging Vision, 2022, pp. 1-21. |
| [20] | J. Liu and H. Zhang, Image segmentation using a local GMM in a variational framework, J. Math. Imaging Vision, 46 (2013), pp. 161-176. · Zbl 1310.94018 |
| [21] | J. M. Lötjönen, R. Wolz, J. R. Koikkalainen, L. Thurfjell, G. Waldemar, H. Soininen, D. Rueckert, and A. D. N. Initiative, Fast and robust multi-atlas segmentation of brain magnetic resonance images, Neuroimage, 49 (2010), pp. 2352-2365. |
| [22] | G. J. McLachlan and T. Krishnan, The EM Algorithm and Extensions, John Wiley & Sons, New York, 2007. · Zbl 1165.62019 |
| [23] | G. J. McLachlan, S. X. Lee, and S. I. Rathnayake, Finite mixture models, Annu. Rev. Stat. Appl., 6 (2019), pp. 355-378. |
| [24] | B. Merriman, J. K. Bence, and S. Osher, Diffusion Generated Motion by Mean Curvature, Department of Mathematics, University of California, Los Angeles, 1992. |
| [25] | B. Merriman, J. K. Bence, and S. Osher, Motion of multiple junctions: A level set approach, J. Comput. Phys., 112 (1994), pp. 334-363. · Zbl 0805.65090 |
| [26] | S. Mesbah, A. M. Shalaby, S. Stills, A. M. Soliman, A. Willhite, S. J. Harkema, E. Rejc, and A. S. El-Baz, Novel stochastic framework for automatic segmentation of human thigh MRI volumes and its applications in spinal cord injured individuals, PLoS One, 14 (2019), e0216487. |
| [27] | M. Molaie and R. A. Zoroofi, A knowledge-based modality-independent technique for concurrent thigh muscle segmentation: Applicable to CT and MR images, J. Digit. Imaging, 33 (2020), pp. 1122-1135. |
| [28] | R. Ni, C. H. Meyer, S. S. Blemker, J. M. Hart, and X. Feng, Automatic segmentation of all lower limb muscles from high-resolution magnetic resonance imaging using a cascaded three-dimensional deep convolutional neural network, J. Med. Imaging, 6 (2019), 044009. |
| [29] | A. Ogier, L. Heskamp, C. P. Michel, A. Fouré, M.-E. Bellemare, A. Le Troter, A. Heerschap, and D. Bendahan, A novel segmentation framework dedicated to the follow-up of fat infiltration in individual muscles of patients with neuromuscular disorders, Magn. Reson. Med., 83 (2020), pp. 1825-1836. |
| [30] | A. Ogier, M. Sdika, A. Foure, A. Le Troter, and D. Bendahan, Individual muscle segmentation in MR images: A 3D propagation through 2D non-linear registration approaches, in Proceedings of the 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2017, pp. 317-320. |
| [31] | A. Pawar, Y. J. Zhang, C. Anitescu, and T. Rabczuk, Joint image segmentation and registration based on a dynamic level set approach using truncated hierarchical B-splines, Comput. Math. Appl., 78 (2019), pp. 3250-3267. · Zbl 1443.65024 |
| [32] | K. M. Pohl, J. Fisher, W. E. L. Grimson, R. Kikinis, and W. M. Wells, A Bayesian model for joint segmentation and registration, NeuroImage, 31 (2006), pp. 228-239. |
| [33] | R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, 1970. · Zbl 0193.18401 |
| [34] | D. Rueckert and P. Aljabar, Non-rigid registration using free-form deformations, in Handbook of Biomedical Imaging, N. Paragios, J. Duncan, and N. Ayache, eds., Springer, New York, 2015, pp. 277-294. |
| [35] | T. W. Sederberg and S. R. Parry, Free-form deformation of solid geometric models, in Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques, ACM, 1986, pp. 151-160. |
| [36] | M. K. Sharma, M. Jas, V. Karale, A. Sadhu, and S. Mukhopadhyay, Mammogram segmentation using multi-atlas deformable registration, Comput. Biol. Med., 110 (2019), pp. 244-253. |
| [37] | A. Sotiras, C. Davatzikos, and N. Paragios, Deformable medical image registration: A survey, IEEE Trans. Med. Imaging, 32 (2013), pp. 1153-1190. |
| [38] | P. Swierczynski, B. W. Papież, J. A. Schnabel, and C. Macdonald, A level-set approach to joint image segmentation and registration with application to CT lung imaging, Comput. Med. Imaging Graph., 65 (2018), pp. 58-68. |
| [39] | A. L. Troter, A. Fouré, M. Guye, S. Confort-Gouny, J.-P. Mattei, J. Gondin, E. Salort-Campana, and D. Bendahan, Volume measurements of individual muscles in human quadriceps femoris using atlas-based segmentation approaches, Magn. Reson. Mater. Phys. Biol. Med., 29 (2016), pp. 245-257. |
| [40] | T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache, Diffeomorphic demons: Efficient non-parametric image registration, NeuroImage, 45 (2009), pp. S61-S72. |
| [41] | V. Vishnevskiy, pTVreg Toolbox, https://github.com/visva89/pTVreg, 2018. |
| [42] | V. Vishnevskiy, T. Gass, G. Székely, and O. Goksel, Total variation regularization of displacements in parametric image registration, in Proceedings of the International MICCAI Workshop on Computational and Clinical Challenges in Abdominal Imaging, Springer, 2014, pp. 211-220. |
| [43] | V. Vishnevskiy, T. Gass, G. Szekely, C. Tanner, and O. Goksel, Isotropic total variation regularization of displacements in parametric image registration, IEEE Trans. Med. Imaging, 36 (2016), pp. 385-395. |
| [44] | P. P. Wyatt and J. A. Noble, MAP MRF joint segmentation and registration of medical images, Med. Image Anal., 7 (2003), pp. 539-552. |
| [45] | A. Yezzi, L. Zollei, and T. Kapur, A variational framework for joint segmentation and registration, in Proceedings of the IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, IEEE, 2001, pp. 44-51. |
| [46] | F. Yokota, Y. Otake, M. Takao, T. Ogawa, T. Okada, N. Sugano, and Y. Sato, Automated muscle segmentation from CT images of the hip and thigh using a hierarchical multi-atlas method, Int. J. Comput. Assist. Radiol. Surg., 13 (2018), pp. 977-986. |
| [47] | B. Zitova and J. Flusser, Image registration methods: A survey, Image Vision Comput., 21 (2003), pp. 977-1000. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
