Feature selection based on the self-calibration of binocular camera extrinsic parameters. (English) Zbl 07776506
Summary: The accuracy of feature-based vision algorithms, including the self-calibration of binocular camera extrinsic parameters used in autonomous driving environment perception techniques relies heavily on the quality of the features extracted from the images. This study investigates the influence of the depth distance between objects and the camera, the feature points in different object regions, and the feature points in dynamic object regions on the self-calibration of binocular camera extrinsic parameters. To achieve this, the study first filters out different types of objects in the image through semantic segmentation. Then, it identifies the areas of dynamic objects and extracts the feature points in the static object region for the self-calibration of binocular camera extrinsic parameters. By calculating the baseline error of the binocular camera and the row alignment error of the matching feature points, this study evaluates the influence of feature points in dynamic object regions, feature points in different object regions, and feature points at different distances on the self-calibration algorithm. The experimental results demonstrate that feature points at static objects close to the camera are beneficial for the self-calibration of extrinsic parameters of binocular camera.
MSC:
| 68T45 | Machine vision and scene understanding |
Software:
SIFTReferences:
| [1] | Alcantarilla, P. F., Yebes, J. J., Almazán, J. and Bergasa, L. M., On combining visual slam and dense scene flow to increase the robustness of localization and mapping in dynamic environments, in IEEE Int. Conf. Robotics and Automation (RiverCentre, Saint Paul, Minnesota, USA, 2012), pp. 1290-1297. |
| [2] | Berg, A. C., Berg, T. L. and Malik, J., Shape matching and object recognition using low distortion correspondences, IEEE Comput. Soc. Conf. Computer Vision and Pattern Recognition (San Diego, CA, USA, 2005), pp. 26-33. |
| [3] | Berry, M. V., Lewis, Z. and Nye, J. F., On the weierstrass-mandelbrot fractal function, Proc. R. Soc. London A Math. Phys. Sci.370(1743) (1980) 459-484. · Zbl 0435.28008 |
| [4] | Bolon-Canedo, V. and Remeseiro, B., Feature selection in image analysis: A survey, Artif. Intell. Rev.53(4) (2020) 2905-2931. |
| [5] | Bouguet, J.-Y., Visual Methods for Three Dimensional Modeling (California Institute of Technology, 1999). |
| [6] | Chen, S., Chen, N., Sun, Z. and Meng, R., Self-calibration and optimization of binocular camera extrinsic parameters, in Int. Conf. Digital Twins and Parallel Intelligence (Beijing, China, 2021), pp. 139-143. |
| [7] | Chen, T., Yuan, H. and Yin, M., Hierarchical unsupervised multi-view feature selection, Int. J. Wavelets, Multiresolution Inf. Process.20(6) (2022) 2250024. · Zbl 1524.68266 |
| [8] | de Almeida, A. M. G., Recco, C. H. and Guido, R. C., Use of paraconsistent feature engineering to support the long term feature choice for speaker verification, Proc. FL AIRS34 (2021). |
| [9] | Dhanachandra, N., Manglem, K. and Chanu, Y. J. J. P. C. S., Image segmentation using k-means clustering algorithm and subtractive clustering algorithm, Proc. Comput. Sci.54 (2015) 764-771. |
| [10] | Fusiello, A., Trucco, E. and Verri, A., A compact algorithm for rectification of stereo pairs, Mach. Vision Appl.12(1) (2000) 16-22. |
| [11] | Gidaris, S. and Komodakis, N., Detect, replace, refine: Deep structured prediction for pixel wise labeling, in Proc. IEEE Conf. Computer Vision and Pattern Recognition (2017), pp. 5248-5257. |
| [12] | Guariglia, E., Harmonic sierpinski gasket and applications, Entropy20(9) (2018) 714. |
| [13] | Guariglia, E., Primality, fractality, and image analysis, Entropy21(3) (2019) 304. · Zbl 1459.26011 |
| [14] | Guariglia, E. and Guido, R. C., Chebyshev wavelet analysis, J. Funct. Spaces (2022) 5542054. |
| [15] | Guariglia, E. and Silvestrov, S., Fractional-wavelet analysis of positive definite distributions and wavelets on \(d^\prime(c)\), Eng. Math. II179 (2016) 337-353. · Zbl 1365.65294 |
| [16] | Guido, R. C., Pedroso, F., Contreras, R. C., Rodrigues, L. C., Guariglia, E. and Neto, J. S., Introducing the discrete path transform (dpt) and its applications in signal analysis, artefact removal, and spoken word recognition, Digital Signal Process.117 (2021) 103158. |
| [17] | Hartley, R. and Zisserman, A., Multiple View Geometry in Computer Vision (Cambridge University Press, 2003). · Zbl 0956.68149 |
| [18] | Karlsson, N., Bernardo, E. D., Ostrowski, J., Goncalves, L., Pirjanian, P. and Munich, M. E., The vslam algorithm for robust localization and mapping, in Proc. IEEE Int. Conf. Robotics and Automation (Barcelona, Spain, 2005), pp. 24-29. |
| [19] | Kendall, A. and Cipolla, R., Geometric loss functions for camera pose regression with deep learning, in Proc. IEEE Conf. Computer Vision and Pattern Recognition (Honolulu, HI, USA, 2017), pp. 5974-5983. |
| [20] | Li, X., Ge, B., Luo, Q., Li, Y. and Tian, Q., Acquisition of camera dynamic extrinsic parameters in free binocular stereo vision system, J. Comput. Appl.37(10) (2017) 2888. |
| [21] | Long, Q., Xie, Q., Mita, S., Ishimaru, K. and Shirai, N., A real-time dense stereo matching method for critical environment sensing in autonomous driving, in Int. IEEE Conf. Intelligent Transportation Systems (Qingdao, China, 2014), pp. 853-860. |
| [22] | Long, Q., Xie, Q., Mita, S., Niknejad, H. T., Ishimaru, K. and Guo, C., Real-time dense disparity estimation based on multi-path viterbi for intelligent vehicle applications, in Proc. British Machine Vision Conference (Nottingham, UK, 2014), pp. 1-13. |
| [23] | Lowe, D. G. J. I. J. o. C. V., Distinctive image features from scale-invariant keypoints, Int. J. Comput. Vision60(2) (2004) 91-110. |
| [24] | Ma, C., Pei, S., Sun, G., Meng, R. and Luo, K., Disparity estimation based on fusion of vision and lidar, Int. J. Wavelets Multiresolution Inf. Process.20(5) (2022) 2250014. · Zbl 1493.62409 |
| [25] | Miksch, M., Yang, B. and Zimmermann, K., Automatic extrinsic camera self-calibration based on homography and epipolar geometry, IEEE Intell. Vehicles Symp. (La Jolla, CA, USA, 2010), pp. 832-839. |
| [26] | Muja, M. and Lowe, D. G., Fast approximate nearest neighbors with automatic algorithm configuration, VISAPP2(331-340) (2009) 2. |
| [27] | Nister, D., An efficient solution to the five-point relative pose problem, IEEE Comput. Soc. Conf. Comput. Vision Pattern Recogn.2 (2003) 22-195. |
| [28] | Parmar, N., Vaswani, A., Uszkoreit, J., Kaiser, L., Shazeer, N., Ku, A. and Tran, D., Image transformer, Int. Conf. Mach. Learning (Stockholm, Sweden, 2018) 4055-4064. |
| [29] | Pereira, F. I., Luft, J. A., Ilha, G. and Susin, A. J. I. T. O. I. T. S., A novel resection-intersection algorithm with fast triangulation applied to monocular visual odometry, IEEE Trans. Intell. Transp. Syst.19(11) (2018) 3584-3593. |
| [30] | Torr, P. H. and Zisserman, A., Feature based methods for structure and motion estimation, in Int. Workshop Vision Algorithms (Corfu, Greece, 1999), pp. 278-294. |
| [31] | Vargas, J., Alsweiss, S., Toker, O., Razdan, R. and Santos, J., An overview of autonomous vehicles sensors and their vulnerability to weather conditions, Sensors21(16) (2021) 5397. |
| [32] | Wall, M. E., Rechtsteiner, A. and Rocha, L. M., Singular value decomposition and principal component analysis, A Practical Approach to Microarray Data Analysis (Springer, Boston, 2003), pp. 91-109. |
| [33] | Wang, Chiao, Sawchuk and Alexander, A., Disparity manipulation for stereo images and video, Stereoscopic Displays Appl.19 (2008) 473-484. |
| [34] | Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X. and Cottrell, G., Understanding convolution for semantic segmentation, IEEE Winter Conf. Applications of Computer Vision (Lake Tahoe, NV, USA, 2018), pp. 1451-1460. |
| [35] | Wang, R., Wan, W., Wang, Y. and Di, K. J. R. S., A new RGB-D slam method with moving object detection for dynamic indoor scenes, Remote Sensing11(10) (2019) 1143. |
| [36] | Wang, Y., Huang, W., Xu, Z. and Wang, M., Adaptive compensation visual odometry in dynamic scenarios, Int. J. Wavelets, Multiresolution Inf. Process.20(4) (2022) 2250003. |
| [37] | Wei, Y., Feng, J., Liang, X., Cheng, M. M., Zhao, Y. and Yan, S., Object region mining with adversarial erasing: A simple classification to semantic segmentation approach, in IEEE Conf. Comput. Vision Pattern Recognition (Honolulu, HI, USA, 2017), pp. 1568-1576. |
| [38] | Xie, Q., Chen, X., Zhang, L., Jiang, A. and Cui, F., A robust and efficient video anti-shaking algorithm for low-end smartphone platforms, IEEE Trans. Consumer Electron. (2019) 1-10. |
| [39] | Xie, Q., Hu, X., Ren, L., Qi, L. and Sun, Z., A binocular vision application in iot: Realtime trustworthy road condition detection system in passable area, IEEE Trans. Ind. Inf.19(1) (2023) 973-983. |
| [40] | Xie, Q., Liu, R., Sun, Z., Pei, S. and Cui, F., A flexible free-space detection system based on stereo vision, Neurocomputing485 (2022) 252-262. |
| [41] | Xie, Q., Long, Q. and Mita, S., Integration of optical flow and multi-path-viterbi algorithm for stereo vision, Int. J. Wavelets, Multiresolution Inf. Process.15(3) (2017) 1750022. · Zbl 1365.68427 |
| [42] | Xu, D., Zhang, D., Liu, X., Ma, L. J. I. T. O. I. and Measurement, A calibration and 3-d measurement method for an active vision system with symmetric yawing cameras, IEEE Trans. Instrum. Measurement70 (2021) 1-13. |
| [43] | Yang, L., Su, H., Zhong, C., Meng, Z. and Lu, Y., Hyperspectral image classification using wavelet transform-based smooth ordering, Int. J. Wavelets Multiresolution Inf. Process.17(10) (2019) 1950050. · Zbl 1434.62140 |
| [44] | Yin, H., Ma, Z., Zhong, M., Wu, K., Wei, Y., Guo, J. and Huang, B. J. S., Slam-based self-calibration of a binocular stereo vision rig in real-time, Sensors20(3) (2020) 621. |
| [45] | Yuan, Y., Chen, X. and Wang, J., Object-contextual representations for semantic segmentation, in European Conf. Computer Vision (Tel Aviv, Israel, 2022), pp. 173-190. |
| [46] | Y. Yuan, L. Huang, J. Guo, C. Zhang, X. Chen and J. Wang, Ocnet: Object context network for scene parsing, preprint (2018), arXiv:1809.00916, pp. 1-22. |
| [47] | Yuan, Y., Xie, J., Chen, X. and Wang, J., Segfix: Model-agnostic boundary refinement for segmentation, in European Conf. Computer Vision (Glasgow, UK, 2020), pp. 489-506. |
| [48] | Zhang, H., Dana, K., Shi, J., Zhang, Z., Wang, X., Tyagi, A. and Agrawal, A., Context encoding for semantic segmentation, in Proc. IEEE Conf. Computer Vision and Pattern Recognition (Salt Lake City, UT, USA, 2018), pp. 7151-7160. |
| [49] | Zhang, L., Li, X., Arnab, A., Yang, K., Tong, Y. and Torr, P., Dual graph convolutional network for semantic segmentation, in Proc. British Machine Vision Conference (Cardiff, UK, 2019), pp. 1-18. |
| [50] | Zhang, Z., A flexible new technique for camera calibration, IEEE Trans. Pattern Anal. Mach. Intelligence22(11) (2000) 1330-1334. |
| [51] | Zhao, S., Chao, M., Liang, W., Ran, M. and Shanshan, P., A deep learning-based binocular perception system, J. Syst. Eng. Electron.32(1) (2021) 7-20. |
| [52] | Zheng, X., Tang, Y. Y. and Zhou, J., A framework of adaptive multiscale wavelet decomposition for signals on undirected graphs, IEEE Trans. Signal Process.67(7) (2019) 1696-1711. · Zbl 1458.94160 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
