Abstract
Conditional Random Field (CRF) is a powerful tool for labeling tasks, and has always played a key role in object recognition and semantic segmentation. However, the quality of CRF labeling depends on selected features, which becomes the bottleneck of the accuracy improvement. In this paper, our semantic segmentation problem is calculated in the same way within the framework of Conditional Random Field. Different from other CRF-based strategies, which use appearance features of image, revealing only little information, we combined our framework together with deep learning strategy, such as Convolutional Neural Networks (CNNs), for feature extraction, which have shown strong ability and remarkable performance. This combination strategy is called deep-feature CRF (dCRF). Through dCRF, the deep informantion of image is illustrated and gets ultilized, and the segmentation accuracy is also increased. The proposed deep CRF strategy is adopted on SIFT-Flow and VOC2007 datasets. The segmentation results reveals that if we use features learned from deep networks into our CRF framework, the performance of our semantic segmentation strategy would increase significantly.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1915–1929 (2013)
Philipp, K., Vladlen, K.: Efficient inference in fully connected CRFs with Gaussian edge potentials. In: Advances in Neural Information Processing Systems, pp. 109–117 (2011)
Jamie, S., John, W., Carsten, R., Antonio, C.: Textonboost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Comput. Vision 81(1), 1–23 (2009)
Li, S.Z., Singh, S.: Markov Random Field Modeling in Image Analysis, vol. 26. Springer, London (2009)
Lafferty, J., McCallum, A., Pereira, F.C.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Eighteenth International Conference on Machine Learning (ICML), pp. 282–289 (2001)
Kae, A., Sohn, K., Lee, H., Learned-Miller, E.: Augmenting CRFs with boltzmann machine shape priors for image labeling. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2019–2026. IEEE (2013)
Cesar, C., Jana, K.: Semantic parsing for priming object detection in RGB-D scenes. In: Semantic Perception, Mapping and Exploration (SPME) (2013)
Cesar, C., Jana, K.: Semantic segmentation with heterogeneous sensor coverages. In: IEEE Robotics and Automation Society (IRAS) (2014)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1. IEEE, pp. 886–893 (2005)
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008)
Liu, F., Lin, G., Shen, C.: CRF learning with CNN features for image segmentation. Pattern Recognit. 48, 2983–2992 (2015)
Sebastian, N., Christoph, H.L.: Structured learning and prediction in computer vision, the essence of knowledge (2011)
Radhakrishna, A., Appu, S., Kevin, S., Aurelien, L., Pascal, F., Sabine, S.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)
Pinheiro, P.H.O., Collobert, R.: Recurrent convolutional neural networks for scene labeling. In: International Conference on Machine Learning (ICML) (2014)
Singh, G., Kosecka, J.: Nonparametric scene parsing with adaptive feature relevance and semantic context. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3151–3157. IEEE (2013)
Eigen, D., Fergus, R.: Nonparametric image parsing using adaptive neighbor sets. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2799–2806. IEEE (2012)
Tighe, J., Lazebnik, S.: SuperParsing: scalable nonparametric image parsing with superpixels. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 352–365. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15555-0_26
Liu, C., Yuen, J., Torralba, A.: Nonparametric scene parsing via label transfer. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2368–2382 (2011)
Everingham, M., VanGool, L., Williams, C., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2014), pp. 580–587. IEEE (2014)
Zhu, J., Mao, J., Yuille, A.: Learning from weakly supervised data by the expectation loss SVM (e-SVM) algorithm. In: Advances in Neural Information Processing Systems (NIPS), pp. 1125–1133 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Yu, H., Song, Y., Ju, W., Liu, Z. (2016). Scene Parsing with Deep Features and Spatial Structure Learning. In: Chen, E., Gong, Y., Tie, Y. (eds) Advances in Multimedia Information Processing - PCM 2016. PCM 2016. Lecture Notes in Computer Science(), vol 9917. Springer, Cham. https://doi.org/10.1007/978-3-319-48896-7_71
Download citation
DOI: https://doi.org/10.1007/978-3-319-48896-7_71
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48895-0
Online ISBN: 978-3-319-48896-7
eBook Packages: Computer ScienceComputer Science (R0)