Skip to main content

Advertisement

SpringerLink
Log in

Fully automatic image segmentation based on FCN and graph cuts

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Due to the posteriori knowledge provided by the Human-Computer Interaction algorithms, interactive segmentation based on Graph Cuts can successfully extract the foreground in an image, which will, however, limit the scope of their application. Inspired by recent years of research on computer vision with fully convolutional neural networks, a fully automated image segmentation method based on FCN and Graph Cuts has been proposed. First, FCN is employed to obtain an original mask for the input image, but its boundary is poor. Second, we generate seed regions heuristically using color histograms and mathematical morphological operations on the original mask. Finally, iterative segmentation is performed using generative seeds and superpixel-level Graph Cuts. Experimental results show that our method has higher segmentation accuracy compared to other representative methods including interactive and automatic segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Boykov, Y., Jolly, M.-P.: Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images, in: Proceedings Eighth IEEE International Conference on Computer Vision. Vancouver, Canada, pp. 105–112 (2001)

  2. Kolmogorov, V., Zabin, R.: What energy functions can be minimized via graph cuts? IEEE Transact. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)

    Article  Google Scholar 

  3. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transact. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  4. Williams, J., Schönlieb, C.-B., Swinfield, T., Lee, J., Cai, X., Qie, L., Coomes, D.A.: 3d segmentation of trees through a flexible multiclass graph cut algorithm. IEEE Transact. Geosci. Remote Sens. 58(2), 754–776 (2020)

    Article  Google Scholar 

  5. Guo, Q., Li, X., Hou, B., Mariethoz, G., Ye, M., Yang, W., Liu, Z.: A novel image reconstruction strategy for ect: combining two algorithms with a graph cut method. IEEE Transact. Instrum. Meas. 69(3), 804���814 (2020)

    Article  Google Scholar 

  6. Rother, C., Kolmogorov, V., Blake, A.: “grabcut’’ interactive foreground extraction using iterated graph cuts. ACM Transact. Graphics (TOG) 23(3), 309–314 (2004)

    Article  Google Scholar 

  7. Blake, A., Rother, C., Brown, M., Perez, P., Torr, P.: Interactive image segmentation using an adaptive gmmrf model, In: European conference on computer vision, Springer, Berlin, pp. 428–441 (2004)

  8. Li, Y., Sun, J., Tang, C.-K., Shum, H.-Y.: Lazy snapping. ACM Transact. Graphics (ToG) 23(3), 303–308 (2004)

    Article  Google Scholar 

  9. An, N.-Y., Pun, C.-M.: Iterated graph cut integrating texture characterization for interactive image segmentation. In: 10th International Conference Computer Graphics. Imaging and Visualization, Macau 2013, 79–83 (2013)

  10. Ren, D., Jia, Z., Yang, J., Kasabov, N.K.: A practical grabcut color image segmentation based on bayes classification and simple linear iterative clustering. IEEE Access 5, 18480–18487 (2017)

    Article  Google Scholar 

  11. Wu, S., Nakao, M., Matsuda, T.: Supercut: Superpixel based foreground extraction with loose bounding boxes in one cutting. IEEE Signal Process Lett 24(12), 1803–1807 (2017)

    Article  Google Scholar 

  12. Long, J., Feng, X., Zhu, X., Zhang, J., Gou, G.: Efficient superpixel-guided interactive image segmentation based on graph theory. Symmetry 10(5), 169 (2018)

    Article  Google Scholar 

  13. Y. Fu, J. Cheng, Z. Li, H. Lu, Saliency cuts: An automatic approach to object segmentation, in: 2008 19th International Conference on Pattern Recognition, Tampa, FL, USA, 2008, pp. 1–4

  14. K.-S. Kim, Y.-J. Yoon, M.-C. Kang, J.-Y. Sun, S.-J. Ko, An improved grabcut using a saliency map, in: 2014 IEEE 3rd Global Conference on Consumer Electronics (GCCE), Tokyo, Japan, 2014, pp. 317–318

  15. S. Li, R. Ju, T. Ren, G. Wu, Saliency cuts based on adaptive triple thresholding, in: 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, 2015, pp. 4609–4613

  16. Cheng, M.-M., Mitra, N.J., Huang, X., Torr, P.H.S., Hu, S.-M.: Global contrast based salient region detection. IEEE Transact. Pattern Anal. Mach. Intell. 37(3), 569–582 (2015)

    Article  Google Scholar 

  17. Niu, Y., Su, C., Guo, W.: Salient object segmentation based on superpixel and background connectivity prior. IEEE Access 6, 56170–56183 (2018)

    Article  Google Scholar 

  18. M. Tang, L. Gorelick, O. Veksler, Y. Boykov, Grabcut in one cut, in: 2013 IEEE International Conference on Computer Vision, Sydney, 2013, pp. 1769–1776

  19. Cheng, M.M., Prisacariu, V.A., Zheng, S., Torr, P.H.S., Rother, C.: Densecut: densely connected crfs for realtime grabcut. Comput Graphics Forum 34(7), 193–201 (2015)

    Article  Google Scholar 

  20. Yong, Z., Jiazheng, Y., Hongzhe, L., Qing, L.: Grabcut image segmentation algorithm based on structure tensor. J China Univ Posts Telecommun 24(2), 38–47 (2017)

    Article  Google Scholar 

  21. S. Vicente, V. Kolmogorov, C. Rother, Joint optimization of segmentation and appearance models, in: 2009 IEEE 12th International Conference on Computer Vision, Kyoto, JAPAN, 2009, pp. 755–762

  22. Gao, Z., Shi, P., Karimi, H.R., Pei, Z.: A mutual grabcut method to solve co-segmentation. EURASIP J. Image Video Proces. 2013(1), 1–11 (2013)

    Article  Google Scholar 

  23. Zhou, H., Zheng, J., Wei, L.: Texture aware image segmentation using graph cuts and active contours. Pattern Recogn. 46(6), 1719–1733 (2013)

    Article  Google Scholar 

  24. Q. Guan, M. Hua, H.-G. Hu, A modified grabcut approach for image segmentation based on local prior distribution, in: 2017 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR), Ningbo, CHINA, 2017, pp. 122–126

  25. H. Yu, Y. Zhou, H. Qian, M. Xian, S. Wang, Loosecut: Interactive image segmentation with loosely bounded boxes, in: 2017 IEEE International Conference on Image Processing (ICIP), Beijing, 2017, pp. 3335–3339

  26. He, K., Wang, D., Tong, M., Zhang, X.: Interactive image segmentation on multiscale appearances. IEEE Access 6, 67732–67741 (2018)

    Article  Google Scholar 

  27. Li, L., Wu, F., Yang, G., Xu, L., Wong, T., Mohiaddin, R., Firmin, D., Keegan, J., Zhuang, X.: Atrial scar quantification via multi-scale cnn in the graph-cuts framework. Med. Image Anal. 60, 101595 (2020)

    Article  Google Scholar 

  28. Dogra, J., Jain, S., Sood, M.: Gradient-based kernel selection technique for tumour detection and extraction of medical images using graph cut. IET Image Processing 14(1), 84–93 (2019)

    Article  Google Scholar 

  29. Hao, M., Zhou, M., Cai, L.: An improved graph-cut-based unsupervised change detection method for multispectral remote-sensing images. Int. J. Remote Sens. 42(11), 4005–4022 (2021)

    Article  Google Scholar 

  30. L. Jia, T. Zhang, J. Fang, F. Dong, Multiple kernel graph cut for sar image change detection. Remote Sens. 13(4):725

  31. Marmanis, D., Wegner, J.D., Galliani, S., Schindler, K., Datcu, M., Stilla, U.: Semantic segmentation of aerial images with an ensemble of cnss. ISPRS Ann. Photogr. Remote Sens. Spatial Inform. Sci. 2016(3), 473–480 (2016)

    Article  Google Scholar 

  32. A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in neural information processing systems, Lake Tahoe, NV, USA, 2012, pp. 1097–1105

  33. K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556

  34. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, 2016, pp. 770–778

  35. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transact. Pattern Anal Mach Intell. 40(4), 834–848 (2017)

    Article  Google Scholar 

  36. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation, in: International Conference on Medical image computing and computer-assisted intervention, Munich, Germany, pp. 234–241 (2015)

  37. Fu, R., Li, B., Gao, Y., Wang, P.: Fully automatic figure-ground segmentation algorithm based on deep convolutional neural network and grabcut. IET Image Proces. 10(12), 937–942 (2016)

    Article  Google Scholar 

  38. Rajchl, M., Lee, M.C.H., Oktay, O., Kamnitsas, K., Passerat-Palmbach, J., Bai, W., Damodaram, M., Rutherford, M.A., Hajnal, J.V., Kainz, B., Rueckert, D.: Deepcut: Object segmentation from bounding box annotations using convolutional neural networks. IEEE Transact. Med. Imaging 36(2), 674–683 (2017)

    Article  Google Scholar 

  39. Yang, Y., Zhuang, Y., Bi, F., Shi, H., Xie, Y.: M-fcn: Effective fully convolutional network-based airplane detection framework. IEEE Geosci. Remote Sens. Lett. 14(8), 1293–1297 (2017)

    Article  Google Scholar 

  40. Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., Torr, P.H.S.: Conditional random fields as recurrent neural networks, santiago, chile, in. IEEE International Conference on Computer Vision (ICCV) 2015, 1529–1537 (2015)

    Article  Google Scholar 

  41. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, USA, pp. 3431–3440 (2015)

  42. Haralick, R.M., Sternberg, S.R., Zhuang, X.: Image analysis using mathematical morphology. IEEE Transact. Pattern Anal. Mach. Intell. PAMI- 9(4), 532–550 (1987)

    Article  Google Scholar 

  43. Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J Comput. Vision 88(2), 303–338 (2010)

    Article  Google Scholar 

  44. Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.-Y.: Learning to detect a salient object. IEEE Transact. Pattern Anal. Mach. Intell 33(2), 353–367 (2010)

    Google Scholar 

  45. Taha, A.A., Hanbury, A.: Metrics for evaluating 3d medical image segmentation: analysis, selection, and tool. BMC Med. Imaging 15(1), 1–28 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

This research is funded by National Natural Science Foundation of China (Grant No. 61201421), National cryosphere desert data center (grant No. E01Z7902) and Capability improvement project for cryosphere desert data center of the Chinese Academy of Sciences (Grant No. Y9298302).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Lanzhou University, Lanzhou, 730000, China

    Zhaobin Wang, Xiong Gao & Runliang Wu

  2. National Cryosphere Desert Data Center,Chinese Academy of Sciences, Lanzhou, 730000, China

    Zhaobin Wang, Jianfang Kang & Yaonan Zhang

  3. Northwest Institute of Eco-Environment and Resources, Chinese Academy of Sciences, Lanzhou, 730000, China

    Zhaobin Wang, Jianfang Kang & Yaonan Zhang

Authors
  1. Zhaobin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiong Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Runliang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianfang Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yaonan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhaobin Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by P. Pala.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Gao, X., Wu, R. et al. Fully automatic image segmentation based on FCN and graph cuts. Multimedia Systems 28, 1753–1765 (2022). https://doi.org/10.1007/s00530-022-00945-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-00945-3

Keywords

Search

Navigation