Abstract
The detection of the vanishing point (VP) in unstructured road is crucial for the advancement of autonomous vehicle technology. However, due to the inadequate fusion of intra-level features and high computational requirements of existing CNN-based road VP detection methods, a model named RVPNet is proposed in this paper. To begin, the proposed algorithm adopts the architecture of encoder-decoder combined lightweight backbone to extract unstructured road features efficiently. Second, the Simple Residual Pyramid Pooling Module (SRPPM) is designed in this model to obtain cross-path global contextual information with low computational cost. And a Dual Attention-based Feature Aggregation Module (DAFAM) is proposed to obtain better inter-level feature representations. Finally, the offset loss is introduced to compensate for the inherent offset errors caused by the output stride of the heatmap. The experimental results show that the average detection error rate of our approach is only 0.03128 on the Kong dataset, and the average processing time reaches 238 FPS. The average detection error rate of our approach based is only 0.03600 on the Moghhadam dataset. Compared with the state-of-the-art methods, the proposed approach achieves the highest detection accuracy and speed.
Similar content being viewed by others
Data Availability
The datasets generated during and analyzed during the current study are available from the corresponding author on reasonable request.
References
Alon Y, Ferencz A, Shashua A (2006) Off-road path following using region classification and geometric projection constraints. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol 1. IEEE, pp 689–696
Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495
Bang S, Park S, Kim H, Kim H (2019) Encoder–decoder network for pixellevel road crack detection in black-box images. Comput-Aided Civil Infrastruct Eng 34(8):713–727
Cai Y, Wang Z, Luo Z, Yin B, Du A, Wang H, Zhang X, Zhou X, Zhou E, Sun J (2020) Learning delicate local representations for multiperson pose estimation. In: Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16. Springer, pp 455–472
Chang C-K, Zhao J, Itti L (2018) Deepvp: Deep learning for vanishing point detection on 1 million street view images. In: 2018 IEEE international conference on robotics and automation (ICRA). IEEE, pp 4496–4503
Chen GH, Zhou W, Wang FJ, Xiao BJ, Dai SF (2013) Lane detection based on improved canny detector and least square fitting. In: Advanced materials research, vol 765. Trans Tech Publ, pp 2383–2387
Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848
Ding W, Li Y (2015) Efficient vanishing point detection method in complex urban road environments. IET Comput Vision 9(4):549–558
Ebrahimpour R, Rasoolinezhad R, Hajiabolhasani Z, Ebrahimi M (2012) Vanishing point detection in corridors: using hough transform and k-means clustering. IET Comput Vision 6(1):40–51
Fei J, Peng K, Heidenreich P, Bieder F, Stiller C (2021) Pillarsegnet: Pillar-based semantic grid map estimation using sparse lidar data. In: 2021 IEEE intelligent vehicles symposium (IV). IEEE, pp 838–844
Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 3146–3154
Huang Z, Wei Y, Wang X, Liu W, Huang TS, Shi H (2021) Alignseg: feature-aligned segmentation networks. IEEE Trans Pattern Anal Mach Intell 44(1):550–557
Huang S, Lu Z, Cheng R, He C (2021) Fapn: feature-aligned pyramid network for dense image prediction. In: Proceedings of the IEEE/CVF international conference on computer vision. pp 864–873
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 7132–7141
Ji Y, Zhang H, Zhang Z, Liu M (2021) Cnn-based encoder-decoder networks for salient object detection: a comprehensive review and recent advances. Inf Sci 546:835–857
Kingma DP, Ba JA, Adam J (2020) A method for stochastic optimization. arxiv 2014. 106. arXiv:1412.6980
Kocur V, Ftáčnik M (2021) Traffic camera calibration via vehicle vanishing point detection. In: Artificial neural networks and machine learning–ICANN 2021: 30th international conference on artificial neural networks, Bratislava, Slovakia, September 14–17, 2021, Proceedings, Part V 30. Springer, pp 628–639
Kogan H, Maurer R, Keshet R (2009). Vanishing points estimation by selfsimilarity. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 755–761
Kong H, Audibert J-Y, Ponce J (2010) General road detection from a single image. IEEE Trans Image Process 19(8):2211–2220
Kong H, Akakin HC, Sarma SE (2013) A generalized laplacian of gaussian filter for blob detection and its applications. IEEE Trans Cybern 43(6):1719–1733
Kortli Y, Marzougui M, Bouallegue B, Bose JSC, Rodrigues P, Atri M (2017) A novel illumination-invariant lane detection system. In: 2017 2nd international conference on anti-cyber crimes (ICACC). IEEE, pp 166– 171
Law H, Deng J (2018) Cornernet: detecting objects as paired keypoints. In: Proceedings of the European conference on computer vision (ECCV). pp 734–750
Lee J, Kim D, Ponce J, Ham B (2019) Sfnet: learning object-aware semantic correspondence. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 2278–2287
Lee S, Kim J, Shin Yoon J, Shin S, Bailo O, Kim N, Lee T- H, Seok Hong H, Han S-H, So Kweon I (2017) Vpgnet: vanishing point guided network for lane and road marking detection and recognition. In: Proceedings of the IEEE international conference on computer vision. pp 1947–1955
Li Y, Tong G, Sun A, Ding W (2018) Road extraction algorithm based on intrinsic image and vanishing point for unstructured road image. Robot Auton Syst 109:86–96
Liu Y-B, Zeng M, Meng Q-H (2020) Unstructured road vanishing point detection using convolutional neural networks and heatmap regression. IEEE Trans Instrum Meas 70:1–8
Liu N, Han J, Yang M-H (2018) Picanet: Learning pixel-wise contextual attention for saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3089–3098
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3431–3440
Moghadam P, Starzyk JA, Wijesoma WS (2011) Fast vanishing-point detection in unstructured environments. IEEE Trans Image Process 21(1):425–430
Newell A, Yang K, Deng J (2016) Stacked hourglass networks for human pose estimation. In: Computer vision–ECCV 2016: 14th European conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14. Springer, pp 483–499
Peng K, Fei J, Yang K, Roitberg A, Zhang J, Bieder F, Heidenreich P, Stiller C, Stiefelhagen R (2022) Mass: multi-attentional semantic segmentation of lidar data for dense top-view understanding. IEEE Trans Intell Transp Syst 23(9):15824–15840
Peng J, Liu Y, Tang S, Hao Y, Chu L, Chen G, Wu Z, Chen Z, Yu Z, Du Y et al. (2022) Pp-liteseg: a superior real-time semantic segmentation model. arXiv:2204.02681
Qin Z, Zhang P, Wu F, Li X (2021) Fcanet: Frequency channel attention networks. In: Proceedings of the IEEE/CVF international conference on computer vision. pp 783–792
Rasmussen C (2008) Roadcompass: following rural roads with vision+ ladar using vanishing point tracking. Auton Robots 25:205–229
Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, pp 234–241
Shen S, Wang S, Wang L, Wei H (2022) A refined-line-based method to estimate vanishing points for vision-based autonomous vehicles. Vehicles 4(2):314–325
Shruthiba A, Deepu R, Dhomne A (2021) A cnn approach to detect vanishing point by considering street lane. In: 2021 IEEE international conference on computation system and information technology for sustainable solutions (CSITSS). pp 1–5. https://doi.org/10.1109/CSITSS54238.2021.9683182
Shuai Y, Tiantian Y, Guodong Y, Zize L (2017) Regression convolutional network for vanishing point detection. In: 2017 32nd youth academic annual conference of Chinese association of automation (YAC). IEEE, pp 634– 638
Song Q, Mei K, Huang R (2021) Attanet: attention-augmented network for fast and accurate scene parsing. Proceedings of the AAAI Conference on Artificial Intelligence 35:2567–2575
Varma G, Subramanian A, Namboodiri A, Chandraker M, Jawahar C (2019) Idd: a dataset for exploring problems of autonomous navigation in unconstrained environments. In: 2019 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1743–1751
Wang C-Y, Bochkovskiy A, Liao H-YM (2022) Yolov7: trainable bag-offreebies sets new state-of-the-art for real-time object detectors. arXiv:2207.02696
Wang Y, Shen D, Teoh EK (2000) Lane detection using spline model. Pattern Recognit Lett 21(8):677–689
Wang Y, Teoh EK, Shen D (2004) Lane detection and tracking using bsnake. Image and Vision Computing 22(4):269–280
Wang P, Fang Z, Zhao S, Chen Y, Zhou M, An S (2021) Vanishing point aided lidar-visual-inertial estimator. In: 2021 IEEE international conference on robotics and automation (ICRA). IEEE, pp 13120–13126
Wang W, Shen J, Dong X, Borji A (2018) Salient object detection driven by fixation prediction. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1711–1720
Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2020) Eca-net: efficient channel attention for deep convolutional neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 11534–11542
Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV). pp 3–19
Wu P-C, Chang C-Y, Lin CH (2014) Lane-mark extraction for automobiles under complex conditions. Pattern Recognit 47(8):2756–2767
Wu Z, Fu W, Xue R, Wang W (2016) A novel line space voting method for vanishing-point detection of general road images. Sensors 16(7):948
Yang G, Wang Y, Yang J, Lu Z (2019) Fast and robust vanishing point detection using contourlet texture detector for unstructured road. IEEE Access 7:139358–139367
Yu C, Xiao B, Gao C, Yuan L, Zhang L, Sang N, Wang J (2021) Lite-hrnet: a lightweight high-resolution network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 10440–10450
Yu Z, Zhu L (2019) Roust vanishing point detection based on the combination of edge and optical flow. In: 2019 4th Asia-pacific conference on intelligent robot systems (ACIRS). pp 184–188. https://doi.org/10.1109/ACIRS.2019.8936016
Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2019) Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39(6):1856–1867
Zhou X, Wang D, Krähenbühl, P (2019) Objects as points. arXiv:1904.07850
Acknowledgements
This research was funded by the Natural Science Foundation of Shandong Province for Key Project under Grant ZR2020KF006, the National Natural Science Foundation of China under Grant 62273164, the Development Program Project of Youth Innovation Team of Institutions of Higher Learning in Shandong Province, and the China Postdoctoral Science Foundation under Grant 2019M662407.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interests/competing interests
The authors have no financial or proprietary interests in any material discussed in this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Yu Liu and Xue Fan contributed equally to this work.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liu, Y., Fan, X., Han, S. et al. RVPNet: A real time unstructured road vanishing point detection algorithm using attention mechanism and global context information. Multimed Tools Appl 83, 28263–28280 (2024). https://doi.org/10.1007/s11042-023-16447-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-16447-x