Deformable models for image segmentation: a critical review of achievements and future challenges. (English) Zbl 1524.65096
Summary: Image segmentation is a fundamental and tedious task of computer vision. Because of inherent noise and intensity inhomogeneity in real-world images, it remains a difficult problem in practical applications such as image analysis, scene understanding, object detection, and many others. Several mathematical models proposed for image segmentation in the past few decades with an effective policy. Among these models, deformable models earned more attention and are widely used techniques due to their accuracy, efficiency, and robust effectiveness. This study reviews and compares various deformable models for the segmentation of digital images available in the literature. First, for a comprehensive study, these deformable models are classified into two classes such as direct partial differential equation (PDE) based approaches and variational based approaches. Beside this, variational based approaches are further classified into parametric and geometric models. Their advantages as well as shortcomings are discussed in detail from an objective viewpoint. Then, to check the robustness of various discussed classes of deformable models, a set of synthetic, natural, and real medical images are considered along with inhomogeneity and noise. Also, to measure the segmentation accuracy, different quantitative metrics based on contour and region are utilized. Numerical experiments with different classes of images, reveal that both direct PDE based and variation based models perform well in clean and noisy images. Whereas, the variational based approaches are superior for images having intensity inhomogeneity with high computational time. The qualitative and quantitative investigations confirm that the subtle change in model assumptions can have a significant impact on segmentation.
MSC:
| 65D18 | Numerical aspects of computer graphics, image analysis, and computational geometry |
| 65D19 | Computational issues in computer and robotic vision |
References:
| [1] | Noble, J. A.; Boukerroui, D., Ultrasound image segmentation: a survey, IEEE Trans. Med. Imaging, 25, 8, 987-1010 (2006) |
| [2] | Shrimali, V.; Anand, R.; Kumar, V., Current trends in segmentation of medical ultrasound b-mode images: a review, IETE Tech. Rev., 26, 1, 8-17 (2009) |
| [3] | Jayadevappa, D.; Srinivas Kumar, S.; Murty, D., Medical image segmentation algorithms using deformable models: a review, IETE Tech. Rev., 28, 3, 248-255 (2011) |
| [4] | Vantaram, S. R.; Saber, E., Survey of contemporary trends in color image segmentation, J. Electron. Imaging, 21, 4, Article 040901 pp. (2012) |
| [5] | Peng, B.; Zhang, L.; Zhang, D., A survey of graph theoretical approaches to image segmentation, Pattern Recognit., 46, 3, 1020-1038 (2013) |
| [6] | Mesejo, P.; Ibáñez, O.; Cordón, O.; Cagnoni, S., A survey on image segmentation using metaheuristic-based deformable models: state of the art and critical analysis, Appl. Soft Comput., 44, 1-29 (2016) |
| [7] | Chouhan, S. S.; Kaul, A.; Singh, U. P., Soft computing approaches for image segmentation: a survey, Multimed. Tools Appl., 77, 21, 28483-28537 (2018) |
| [8] | Yu, H.; He, F.; Pan, Y., A survey of level set method for image segmentation with intensity inhomogeneity, Multimed. Tools Appl., 79, 39, 28525-28549 (2020) |
| [9] | Wang, Z.; Ma, B.; Zhu, Y., Review of level set in image segmentation, Arch. Comput. Methods Eng., 1-18 (2020) |
| [10] | Kaur, A.; Kaur, L.; Singh, A., State-of-the-art segmentation techniques and future directions for multiple sclerosis brain lesions, Arch. Comput. Methods Eng., 1-27 (2020) |
| [11] | Biswas, S.; Hazra, R., State-of-the-art level set models and their performances in image segmentation: a decade review, Arch. Comput. Methods Eng., 1-24 (2021) |
| [12] | Otsu, N., A threshold selection method from gray-level histograms, IEEE Trans. Syst. Man Cybern., 9, 1, 62-66 (1979) |
| [13] | Sezgin, M.; Sankur, B., Survey over image thresholding techniques and quantitative performance evaluation, J. Electron. Imaging, 13, 1, 146-165 (2004) |
| [14] | Bhardwaj, S.; Mittal, A., A survey on various edge detector techniques, Proc. Technol., 4, 220-226 (2012) |
| [15] | Canny, J., A computational approach to edge detection, IEEE Trans. Pattern Anal. Mach. Intell., 6, 679-698 (1986) |
| [16] | Najman, L.; Schmitt, M., Watershed of a continuous function, Signal Process., 38, 1, 99-112 (1994) |
| [17] | Boykov, Y.; Funka-Lea, G., Graph cuts and efficient nd image segmentation, Int. J. Comput. Vis., 70, 2, 109-131 (2006) |
| [18] | Chen, C. W.; Luo, J.; Parker, K. J., Image segmentation via adaptive k-mean clustering and knowledge-based morphological operations with biomedical applications, IEEE Trans. Image Process., 7, 12, 1673-1683 (1998) |
| [19] | Yang, Z.; Chung, F.-L.; Shitong, W., Robust fuzzy clustering-based image segmentation, Appl. Soft Comput., 9, 1, 80-84 (2009) |
| [20] | Gómez, D.; Yanez, J.; Guada, C.; Rodríguez, J. T.; Montero, J.; Zarrazola, E., Fuzzy image segmentation based upon hierarchical clustering, Knowl.-Based Syst., 87, 26-37 (2015) |
| [21] | Adams, R.; Bischof, L., Seeded region growing, IEEE Trans. Pattern Anal. Mach. Intell., 16, 6, 641-647 (1994) |
| [22] | Strasters, K. C.; Gerbrands, J. J., Three-dimensional image segmentation using a split, merge and group approach, Pattern Recognit. Lett., 12, 5, 307-325 (1991) |
| [23] | Seo, H.; Badiei Khuzani, M.; Vasudevan, V.; Huang, C.; Ren, H.; Xiao, R.; Jia, X.; Xing, L., Machine learning techniques for biomedical image segmentation: an overview of technical aspects and introduction to state-of-art applications, Med. Phys., 47, 5, e148-e167 (2020) |
| [24] | Pratondo, A.; Chui, C.-K.; Ong, S.-H., Integrating machine learning with region-based active contour models in medical image segmentation, J. Vis. Commun. Image Represent., 43, 1-9 (2017) |
| [25] | Wu, X.; Tan, G.; Li, K.; Li, S.; Wen, H.; Zhu, X.; Cai, W., Deep parametric active contour model for neurofibromatosis segmentation, Future Gener. Comput. Syst., 112, 58-66 (2020) |
| [26] | Osher, S.; Sethian, J. A., Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations, J. Comput. Phys., 79, 1, 12-49 (1988) · Zbl 0659.65132 |
| [27] | Caselles, V.; Catté, F.; Coll, T.; Dibos, F., A geometric model for active contours in image processing, Numer. Math., 66, 1, 1-31 (1993) · Zbl 0804.68159 |
| [28] | Malladi, R.; Sethian, J. A.; Vemuri, B. C., Shape modeling with front propagation: a level set approach, IEEE Trans. Pattern Anal. Mach. Intell., 17, 2, 158-175 (1995) |
| [29] | Paragios, N.; Mellina-Gottardo, O.; Ramesh, V., Gradient vector flow fast geodesic active contours, (Proceedings Eighth IEEE International Conference on Computer Vision, ICCV 2001, vol. 1 (2001), IEEE), 67-73 |
| [30] | Xu, C.; Prince, J. L., Snakes, shapes, and gradient vector flow, IEEE Trans. Image Process., 7, 3, 359-369 (1998) · Zbl 0973.94003 |
| [31] | Caselles, V.; Kimmel, R.; Sapiro, G., Geodesic active contours, Int. J. Comput. Vis., 22, 1, 61-79 (1997) · Zbl 0894.68131 |
| [32] | Goldenberg, R.; Kimmel, R.; Rivlin, E.; Rudzsky, M., Fast geodesic active contours, IEEE Trans. Image Process., 10, 10, 1467-1475 (2001) |
| [33] | Weickert, J.; Kühne, G., Fast methods for implicit active contour models, (Geometric Level Set Methods in Imaging, Vision, and Graphics (2003), Springer), 43-57 |
| [34] | Zhang, K.; Zhang, L.; Song, H.; Zhou, W., Active contours with selective local or global segmentation: a new formulation and level set method, Image Vis. Comput., 28, 4, 668-676 (2010) |
| [35] | Tran, T.-T.; Pham, V.-T.; Chiu, Y.-J.; Shyu, K.-K., Active contour with selective local or global segmentation for intensity inhomogeneous image, (2010 3rd International Conference on Computer Science and Information Technology, vol. 1 (2010), IEEE), 306-310 |
| [36] | Zhang, K.; Zhang, L.; Song, H.; Zhang, D., Reinitialization-free level set evolution via reaction diffusion, IEEE Trans. Image Process., 22, 1, 258-271 (2012) · Zbl 1373.94483 |
| [37] | Talu, M. F., ORACM: online region-based active contour model, Expert Syst. Appl., 40, 16, 6233-6240 (2013) |
| [38] | Sun, L.; Meng, X.; Xu, J.; Tian, Y., An image segmentation method using an active contour model based on improved spf and lif, Appl. Sci., 8, 12, 2576 (2018) |
| [39] | Zhang, K.; Song, H.; Zhang, L., Active contours driven by local image fitting energy, Pattern Recognit., 43, 4, 1199-1206 (2010) · Zbl 1192.68624 |
| [40] | Hanbay, K.; Talu, M. F., A novel active contour model for medical images via the Hessian matrix and eigenvalues, Comput. Math. Appl., 75, 9, 3081-3104 (2018) · Zbl 1409.92131 |
| [41] | Wang, Y.; Yuan, Q.; He, C., Indirect diffusion based level set evolution for image segmentation, Appl. Math. Model., 69, 714-722 (2019) · Zbl 1466.65021 |
| [42] | Kass, M.; Witkin, A.; Terzopoulos, D., Snakes: active contour models, Int. J. Comput. Vis., 1, 4, 321-331 (1988) |
| [43] | Cohen, L. D., On active contour models and balloons, CVGIP, Image Underst., 53, 2, 211-218 (1991) · Zbl 0774.68111 |
| [44] | Cohen, L. D.; Cohen, I., Finite-element methods for active contour models and balloons for 2-d and 3-d images, IEEE Trans. Pattern Anal. Mach. Intell., 15, 11, 1131-1147 (1993) |
| [45] | Poon, C. S.; Braun, M.; Fahrig, R.; Ginige, A.; Dorrell, A., Segmentation of medical images using an active-contour model incorporating region-based image features, (Visualization in Biomedical Computing 1994, vol. 2359 (1994), International Society for Optics and Photonics), 90-97 |
| [46] | Liu, T.; Zhou, H.; Lin, F.; Pang, Y.; Wu, J., Improving image segmentation by gradient vector flow and mean shift, Pattern Recognit. Lett., 29, 1, 90-95 (2008) |
| [47] | Xu, C.; Prince, J. L., Generalized gradient vector flow external forces for active contours, Signal Process., 71, 2, 131-139 (1998) · Zbl 0908.68202 |
| [48] | Sum, K.; Cheung, P. Y., Boundary vector field for parametric active contours, Pattern Recognit., 40, 6, 1635-1645 (2007) · Zbl 1114.68061 |
| [49] | Jacob, M.; Blu, T.; Unser, M., Efficient energies and algorithms for parametric snakes, IEEE Trans. Image Process., 13, 9, 1231-1244 (2004) |
| [50] | Hou, Z.; Han, C., Force field analysis snake: an improved parametric active contour model, Pattern Recognit. Lett., 26, 5, 513-526 (2005) |
| [51] | Cvancarova, M.; Albregtsen, F.; Brabrand, K.; Samset, E., Segmentation of ultrasound images of liver tumors applying snake algorithms and GVF, (International Congress Series, vol. 1281 (2005), Elsevier), 218-223 |
| [52] | Zimmer, C.; Olivo-Marin, J.-C., Coupled parametric active contours, IEEE Trans. Pattern Anal. Mach. Intell., 27, 11, 1838-1842 (2005) |
| [53] | Chan, T. F.; Vese, L. A., Active contours without edges, IEEE Trans. Image Process., 10, 2, 266-277 (2001) · Zbl 1039.68779 |
| [54] | Jifeng, N.; Chengke, W.; Shigang, L.; Shuqin, Y., NGVF: an improved external force field for active contour model, Pattern Recognit. Lett., 28, 1, 58-63 (2007) |
| [55] | Wang, Y.; Jia, Y.; Liu, L., Harmonic gradient vector flow external force for snake model, Electron. Lett., 44, 2, 105-106 (2008) |
| [56] | Tauber, C.; Batatia, H.; Ayache, A., Quasi-automatic initialization for parametric active contours, Pattern Recognit. Lett., 31, 1, 83-90 (2010) |
| [57] | Qin, L.; Zhu, C.; Zhao, Y.; Bai, H.; Tian, H., Generalized gradient vector flow for snakes: new observations, analysis, and improvement, IEEE Trans. Circuits Syst. Video Technol., 23, 5, 883-897 (2013) |
| [58] | Wu, Y.; Wang, Y.; Jia, Y., Adaptive diffusion flow active contours for image segmentation, Comput. Vis. Image Underst., 117, 10, 1421-1435 (2013) |
| [59] | Zhang, Z.; Duan, C.; Lin, T.; Zhou, S.; Wang, Y.; Gao, X., GVFOM: a novel external force for active contour based image segmentation, Inf. Sci., 506, 1-18 (2020) · Zbl 1456.94013 |
| [60] | Mumford, D. B.; Shah, J., Optimal approximations by piecewise smooth functions and associated variational problems, Commun. Pure Appl. Math. (1989) · Zbl 0691.49036 |
| [61] | Paragios, N.; Deriche, R., Geodesic active contours and level sets for the detection and tracking of moving objects, IEEE Trans. Pattern Anal. Mach. Intell., 22, 3, 266-280 (2000) |
| [62] | Tsai, A.; Yezzi, A.; Willsky, A. S., Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolation, and magnification, IEEE Trans. Image Process., 10, 8, 1169-1186 (2001) · Zbl 1062.68595 |
| [63] | Vese, L. A.; Chan, T. F., A multiphase level set framework for image segmentation using the Mumford and Shah model, Int. J. Comput. Vis., 50, 3, 271-293 (2002) · Zbl 1012.68782 |
| [64] | Li, C.; Xu, C.; Gui, C.; Fox, M. D., Level set evolution without re-initialization: a new variational formulation, (2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1 (2005), IEEE), 430-436 |
| [65] | Lee, S.-H.; Seo, J. K., Level set-based bimodal segmentation with stationary global minimum, IEEE Trans. Image Process., 15, 9, 2843-2852 (2006) |
| [66] | Li, C.; Kao, C.-Y.; Gore, J. C.; Ding, Z., Implicit active contours driven by local binary fitting energy, (2007 IEEE Conference on Computer Vision and Pattern Recognition (2007), IEEE), 1-7 |
| [67] | Li, C.; Kao, C.-Y.; Gore, J. C.; Ding, Z., Minimization of region-scalable fitting energy for image segmentation, IEEE Trans. Image Process., 17, 10, 1940-1949 (2008) · Zbl 1371.94225 |
| [68] | Krinidis, S.; Chatzis, V., Fuzzy energy-based active contours, IEEE Trans. Image Process., 18, 12, 2747-2755 (2009) · Zbl 1371.94200 |
| [69] | Wang, X.-F.; Huang, D.-S.; Xu, H., An efficient local Chan-Vese model for image segmentation, Pattern Recognit., 43, 3, 603-618 (2010) · Zbl 1185.68817 |
| [70] | Liu, S.; Peng, Y., A local region-based Chan-Vese model for image segmentation, Pattern Recognit., 45, 7, 2769-2779 (2012) · Zbl 1236.68275 |
| [71] | Li, C.; Xu, C.; Gui, C.; Fox, M. D., Distance regularized level set evolution and its application to image segmentation, IEEE Trans. Image Process., 19, 12, 3243-3254 (2010) · Zbl 1371.94226 |
| [72] | Li, C.; Huang, R.; Ding, Z.; Gatenby, J. C.; Metaxas, D. N.; Gore, J. C., A level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI, IEEE Trans. Image Process., 20, 7, 2007-2016 (2011) · Zbl 1372.68275 |
| [73] | Chen, B.; Li, Y.; Cai, J.-l., Noisy image segmentation based on nonlinear diffusion equation model, Appl. Math. Model., 36, 3, 1197-1208 (2012) · Zbl 1243.94006 |
| [74] | Badshah, N.; Chen, K.; Ali, H.; Murtaza, G., Coefficient of variation based image selective segmentation model using active contours, East Asian J. Appl. Math., 2, 2, 150-169 (2012) · Zbl 1284.68623 |
| [75] | Li, Y.; Kim, J., An unconditionally stable numerical method for bimodal image segmentation, Appl. Math. Comput., 219, 6, 3083-3090 (2012) · Zbl 1309.65026 |
| [76] | Dong, F.; Chen, Z.; Wang, J., A new level set method for inhomogeneous image segmentation, Image Vis. Comput., 31, 10, 809-822 (2013) |
| [77] | Bai, X.; Wang, W., Saliency snake: a unified framework with adaptive initial curve, Optik, 125, 23, 6972-6976 (2014) |
| [78] | Wu, Y.; He, C., A convex variational level set model for image segmentation, Signal Process., 106, 123-133 (2015) |
| [79] | Fredo, A.; Kavitha, G.; Ramakrishnan, S., Segmentation and analysis of corpus callosum in autistic MR brain images using reaction diffusion level sets, J. Med. Imag. Health Inform., 5, 4, 737-741 (2015) |
| [80] | Wu, Y.; Ma, W.; Gong, M.; Li, H.; Jiao, L., Novel fuzzy active contour model with kernel metric for image segmentation, Appl. Soft Comput., 34, 301-311 (2015) |
| [81] | Ren, Z., Adaptive active contour model driven by fractional order fitting energy, Signal Process., 117, 138-150 (2015) |
| [82] | Gu, M.; Wang, R., Fractional differentiation-based active contour model driven by local intensity fitting energy, Math. Probl. Eng., 2016 (2016) |
| [83] | Lv, H.; Wang, Z.; Fu, S.; Zhang, C.; Zhai, L.; Liu, X., A robust active contour segmentation based on fractional-order differentiation and fuzzy energy, IEEE Access, 5, 7753-7761 (2017) |
| [84] | Ding, K.; Xiao, L.; Weng, G., Active contours driven by region-scalable fitting and optimized Laplacian of Gaussian energy for image segmentation, Signal Process., 134, 224-233 (2017) |
| [85] | Ding, K.; Xiao, L.; Weng, G., Active contours driven by local pre-fitting energy for fast image segmentation, Pattern Recognit. Lett., 104, 29-36 (2018) |
| [86] | Liu, J.; Wen, X.; Meng, Q.; Xu, H.; Yuan, L., Synthetic aperture radar image segmentation with reaction diffusion level set evolution equation in an active contour model, Remote Sens., 10, 6, 906 (2018) |
| [87] | Liu, Y.; He, C.; Wu, Y., Variational model with kernel metric-based data term for noisy image segmentation, Digit. Signal Process., 78, 42-55 (2018) |
| [88] | Merriman, B.; Bence, J. K.; Osher, S. J., Motion of multiple junctions: a level set approach, J. Comput. Phys., 112, 2, 334-363 (1994) · Zbl 0805.65090 |
| [89] | Ibrahim, R. W.; Hasan, A. M.; Jalab, H. A., A new deformable model based on fractional Wright energy function for tumor segmentation of volumetric brain MRI scans, Comput. Methods Programs Biomed., 163, 21-28 (2018) |
| [90] | Chen, B.; Huang, S.; Liang, Z.; Chen, W.; Pan, B., A fractional order derivative based active contour model for inhomogeneous image segmentation, Appl. Math. Model., 65, 120-136 (2019) · Zbl 1480.94005 |
| [91] | Ala’a, R.; Jalab, H. A.; Shivakumara, P.; Ibrahim, R. W.; Obaidellah, U. H., Kidney segmentation in MR images using active contour model driven by fractional-based energy minimization, Signal Image Video Process., 14, 7, 1361-1368 (2020) |
| [92] | Lv, H.; Zhang, F.; Wang, R., Fuzzy active contour model using fractional-order diffusion based edge indicator and fuzzy local fitted image, IEEE Access, 8, 172707-172722 (2020) |
| [93] | Li, M.-M.; Li, B.-Z., A novel active contour model for noisy image segmentation based on adaptive fractional order differentiation, IEEE Trans. Image Process., 29, 9520-9531 (2020) · Zbl 07586565 |
| [94] | Yan, X.; Weng, G., Hybrid active contour model driven by optimized local pre-fitting image energy for fast image segmentation, Appl. Math. Model. (2021) · Zbl 1481.94042 |
| [95] | Alpert, S.; Galun, M.; Basri, R.; Brandt, A., Image segmentation by probabilistic bottom-up aggregation and cue integration, (2007 IEEE Conference on Computer Vision and Pattern Recognition (2007)), 1-8 |
| [96] | Karunanayake, N.; Aimmanee, P.; Lohitvisate, W.; Makhanov, S. S., Particle method for segmentation of breast tumors in ultrasound images, Math. Comput. Simul., 170, 257-284 (2020) · Zbl 1510.92114 |
| [97] | Chicco, D.; Jurman, G., The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation, BMC Genomics, 21, 1, 1-13 (2020) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
