Abstract
Salient object detection remains one of the most important and active research topics in computer vision, with wide-ranging applications to object recognition, scene understanding, image retrieval, context aware image editing, image compression, etc. Most existing methods directly determine salient objects by exploring various salient object features. Here, we propose a novel graph based ranking method to detect and segment the most salient object in a scene according to its relationship to image border (background) regions, i.e., the background feature. Firstly, we use regions/super-pixels as graph nodes, which are fully connected to enable both long range and short range relations to be modeled. The relationship of each region to the image border (background) is evaluated in two stages: (i) ranking with hard background queries, and (ii) ranking with soft foreground queries. We experimentally show how this two-stage ranking based salient object detection method is complementary to traditional methods, and that integrated results outperform both. Our method allows the exploitation of intrinsic image structure to achieve high quality salient object determination using a quadratic optimization framework, with a closed form solution which can be easily computed. Extensive method evaluation and comparison using three challenging saliency datasets demonstrate that our method consistently outperforms 10 state-of-theart models by a big margin.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Borji, A.; Itti, L. State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 1, 185–207, 2013.
Toet, A. Computational versus psychophysical bottom–up image saliency: A comparative evaluation study. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 11, 2131–2146, 2011.
Itti, L. Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Transactions on Image Processing Vol. 13, No. 10, 1304–1318, 2004.
Rother, C.; Kolmogorov, V.; Blake, A. “GrabCut”: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics Vol. 23, No.3, 309–314, 2004.
Sharma, G.; Jurie, F.; Schmid, C. Discriminative spatial saliency for image classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3506–3513, 2012.
Avidan, S.; Shamir, A. Seam carving for content-aware image resizing. ACM Transactions on Graphics Vol. 26, No. 3, Article No. 10, 2007.
Zhang, G.-X.; Cheng, M.-M.; Hu, S.-M.; Martin, R. R. A shape-preserving approach to image resizing. Computer Graphics Forum Vol. 28, No. 7, 1897–1906, 2009.
Zhang, L.; Huang, H. Hierarchical narrative collage for digital photo album. Computer Graphics Forum Vol. 31, No. 7, 2173–2181, 2012.
Shen, C.; Zhao, Q. Webpage saliency. In: Lecture Notes in Computer Science, Vol. 8695. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer International Publishing, 33–46, 2014.
Mahadevan, V.; Vasconcelos, N. Saliency-based discriminant tracking. In: Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, 1007–1013, 2009.
Shen, X.; Wu, Y. A unified approach to salient object detection via low rank matrix recovery. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 853–860, 2012.
Yang, C.; Zhang, L.; Lu, H.; Ruan, X.; Yang, M.-H. Saliency detection via graph-based manifold ranking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3166–3173, 2013.
Li, X.; Lu, H.; Zhang, L.; Ruan, X.; Yang, M.-H. Saliency detection via dense and sparse reconstruction. In: Proceedings of IEEE International Conference on Computer Vision, 2976–2983, 2013.
Achanta, R.; Hemami, S.; Estrada, F.; Süsstrunk, S. Frequency-tuned salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1597–1604, 2009.
Cheng, M.-M.; Mitra, N. J.; Huang, X.; Torr, P. H. S.; Hu, S.-M. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 37, No. 3, 569–582, 2015.
Liu, T.; Yuan, Z.; Sun, J.; Wang, J.; Zheng, N.; Tang, X.; Shum, H.-Y. Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 2, 353–367, 2011.
Yan, Q.; Xu, L.; Shi, J.; Jia, J. Hierarchical saliency detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1155–1162, 2013.
Borji, A.; Sihite, D. N.; Itti, L. Salient object detection: A benchmark. In: Proceedings of the 12th European Conference on Computer Vision, Vol. II, 414–429, 2012.
Itti, L.; Koch, C.; Niebur, E. A model of saliencybased visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 20, No. 11, 1254–1259, 1998.
Harel, J.; Koch, C.; Perona, P. Graph-based visual saliency. In: Proceedings of Advances in Neural Information Processing Systems 19, 2006. Available at http://papersnipscc/paper/3095-graphbased-visual-saliencypdf.
Hou, X.; Zhang, L. Saliency detection: A spectral residual approach. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1–8, 2007.
Zhang, L.; Tong, M. H.; Marks, T. K.; Shan, H.; Cottrell, G. W. SUN: A Bayesian framework for saliency using natural statistics. Journal of Vision Vol. 8, No. 7, Article No. 32, 2008.
Rahtu, E.; Kannala, J.; Salo, M.; Heikkilä, J. Segmenting salient objects from images and videos. In: Proceedings of the 11th European Conference on Computer Vision: Part V, 366–379, 2010.
Goferman, S.; Zelnik-Manor, L.; Tal, A. Contextaware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 10, 1915–1926, 2012.
Jiang, B.; Zhang, L.; Lu, H.; Yang, C.; Yang, M.-H. Saliency detection via absorbing Markov chain. In: Proceedings of IEEE International Conference on Computer Vision, 1665–1672, 2013.
Jiang, P.; Ling, H.; Yu, J.; Peng, J. Salient region detection by UFO: Uniqueness, focusness and objectness. In: Proceedings of IEEE International Conference on Computer Vision, 1976–1983, 2013.
Jiang, H.; Wang, J.; Yuan, Z.; Wu, Y.; Zheng, N.; Li, S. Salient object detection: A discriminative regional feature integration approach. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2083–2090, 2013.
Li, X.; Li, Y.; Shen, C.; Dick, A.; Hengel, A. V. D. Contextual hypergraph modeling for salient object detection. In: Proceedings of IEEE International Conference on Computer Vision, 3328–3335, 2013.
Zhang, J.; Sclaroff, S. Saliency detection: A Boolean map approach. In: Proceedings of IEEE International Conference on Computer Vision, 153–160, 2013.
Kim, J.; Han, D.; Tai, Y.-W.; Kim, J. Salient region detection via high-dimensional color transform. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 883–890, 2014.
Jiang, Z.; Davis, L. S. Submodular salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2043–2050, 2013.
Margolin, R.; Tal, A.; Zelnik-Manor, L. What makes a patch distinct? In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1139–1146, 2013.
Liu, R.; Cao, J.; Lin, Z.; Shan, S. Adaptive partial differential equation learning for visual saliency detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3866–3873, 2014.
Li, N.; Ye, J.; Ling, H.; Yu, J. Saliency detection on light field. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2806–2813, 2014.
Jiang, M.; Huang, S.; Duan, J.; Zhao, Q. SALICON: Saliency in context. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1072–1080, 2015.
Cheng, M.-M.; Mitra, N. J.; Huang, X.; Hu, S.-M. SalientShape: Group saliency in image collections. The Visual Computer Vol. 30, No. 4, 443–453, 2014.
Fu, H.; Cao, X.; Tu, Z. Cluster-based co-saliency detection. IEEE Transactions on Image Processing Vol. 22, No. 10, 3766–3778, 2013.
Wei, Y.; Wen, F.; Zhu, W.; Sun, J. Geodesic saliency using background priors. In: Proceedings of the 12th European Conference on Computer Vision, Vol. III, 29–42, 2012.
Zhu, W.; Liang, S.; Wei, Y.; Sun, J. Saliency optimization from robust background detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2814–2821, 2014.
Zhou, D.; Weston, J.; Gretton, A.; Bousquet, O.; Schölkopf, B. Ranking on data manifolds. In: Proceedings of Advances in Neural Information Processing Systems 16, 2004. Available at http://papersnipscc/paper/2447-ranking-on-datamanifolds. pdf.
Achanta, R.; Shaji, A.; Smith, K.; Lucchi, A.; Fua, P.; Süsstrunk, S. SLIC superpixels compared to stateof-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 11, 2274–2282, 2012.
Jiang, H.; Wang, J.; Yuan, Z.; Liu, T.; Zheng, N. Automatic salient object segmentation based on context and shape prior. In: Proceedings of the British Machine Vision Conference, 110.1–12, 2011.
Canny, J. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 8, No. 6, 679–698, 1986.
Krähenbühl, P.; Koltun, V. Efficient inference in fully connected CRFs with Gaussian edge potentials. In: Proceedings of Advances in Neural Information Processing Systems 24, 2011. Available at http://papersnipscc/paper/4296-efficientinference-in-fully-connected-crfs-with-gaussian-edgepotentials. pdf.
Duan, L.; Wu, C.; Miao, J.; Qing, L.; Fu, Y. Visual saliency detection by spatially weighted dissimilarity. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 473–480, 2011.
Perazzi, F.; Krahenbuhl, P.; Pritch, Y.; Hornung, A. Saliency filters: Contrast based filtering for salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 733–740, 2012.
Cheng, M.-M.; Warrell, J.; Lin, W.-Y.; Zheng, S.; Vineet, V.; Crook, N. Efficient salient region detection with soft image abstraction. In: Proceedings of IEEE International Conference on Computer Vision, 1529–1536, 2013.
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Wei Qi is currently a Ph.D. candidate at Nanjing University of Science and Technology. His research interests include visual attention, object detection, and image enhancement.
Ming-Ming Cheng received his Ph.D. degree from Tsinghua University in 2012. Then he was a two-year research fellow, with Prof. Philip Torr in Oxford. He is now an associate professor at Nankai University. His research interests include computer graphics, computer vision, and image processing.
Ali Borji received his Ph.D. degree in cognitive neurosciences at Institute for Studies in Fundamental Sciences (IPM) in Tehran, Iran, 2009, and spent four years as a postdoctoral scholar at iLab, University of Southern California from 2010 to 2014. He is currently an assistant professor at University of Wisconsin, Milwaukee, USA. His research interests include visual attention, active learning, object and scene recognition, and cognitive and computational neurosciences.
Huchuan Lu received his M.S. degree in signal and information processing and Ph.D. degree in system engineering from Dalian University of Technology (DUT), China, in 1998 and 2008, respectively. He joined DUT in 1998, as a faculty member, where he is currently a full professor with the School of Information and Communication Engineering. His research interests include visual tracking, saliency detection, and segmentation. He is a member of the Association for Computing Machinery, and an Associate Editor of IEEE Transactions on Cybernetics.
Lian-Fa Bai is a professor of Jiangsu Key Laboratory of Spectral Imaging and Intelligence Sense, Nanjing University of Science and Technology. He got his Ph.D. degree in Nanjing University of Science and Technology. His current research interests include computer vision and image detection.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Qi, W., Cheng, MM., Borji, A. et al. SaliencyRank: Two-stage manifold ranking for salient object detection. Comp. Visual Media 1, 309–320 (2015). https://doi.org/10.1007/s41095-015-0028-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-015-0028-y