A 2D/3D model-based object tracking framework. (English) Zbl 1047.68622
Summary: This paper presents a robust framework for tracking complex objects in video sequences. Multiple hypothesis tracking (MHT) algorithm reported in [IEEE Trans. Pattern Anal. Mach. Intell. 18, No. 2 (1996)] is modified to accommodate a high level representations (2D edge map, 3D models) of objects for tracking. The framework exploits the advantages of MHT algorithm which is capable of resolving data association/uncertainty and integrates it with object matching techniques to provide a robust behavior while tracking complex objects. To track objects in 2D, a 4D feature is used to represent edge/line segments and are tracked using MHT. In many practical applications 3D models provide more information about the object’s pose (i.e., rotation information in the transformation space) which cannot be recovered using 2D edge information. Hence, a 3D model-based object tracking algorithm is also presented. A probabilistic Hausdorff image matching algorithm is incorporated into the framework in order to determine the geometric transformation that best maps the model features onto their corresponding ones in the image plane. 3D model of the object is used to constrain the tracker to operate in a consistent manner. Experimental results on real and synthetic image sequences are presented to demonstrate the efficacy of the proposed framework.
References:
| [1] | Cox, I. J.; Hingorani, S. L., An efficient implementation of Reid’s multiple hypothesis tracking algorithm and its evaluation for the purpose of visual tracking, IEEE Trans. Pattern Anal. Mach. Intell., 18, 2, 138-150 (1996) |
| [2] | J. MacCormick, A. Blake, A probabilistic exclusion principle for tracking multiple objects, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Corfu, Greece, 1999, 572-578.; J. MacCormick, A. Blake, A probabilistic exclusion principle for tracking multiple objects, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Corfu, Greece, 1999, 572-578. · Zbl 1060.68629 |
| [3] | Rasmussen, C.; Hager, G. D., Probabilistic data association methods for tracking complex visual objects, IEEE Trans. Pattern Anal. Mach. Intell., 23, 6, 560-576 (2001) |
| [4] | Haritaoglu, I.; Harwood, D.; Davis, L., W4real-time surveillance of people and their activities, IEEE Trans. Pattern Anal. Mach. Intell., 22, 8, 809-830 (2000) |
| [5] | E. Polat, M. Yeasin, R. Sharma, Tracking body parts of multiple people: a new approach, in: IEEE International Conference on Computer Vision, Workshop on Multi Object Tracking, Vancouver, Canada, 2001, 35-42.; E. Polat, M. Yeasin, R. Sharma, Tracking body parts of multiple people: a new approach, in: IEEE International Conference on Computer Vision, Workshop on Multi Object Tracking, Vancouver, Canada, 2001, 35-42. |
| [6] | S. Intille, J. Davis, A. Bobick, Real-time closed-world tracking, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997, San Juan, Puerto Rico, pp. 697-703.; S. Intille, J. Davis, A. Bobick, Real-time closed-world tracking, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997, San Juan, Puerto Rico, pp. 697-703. |
| [7] | W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee, Using adaptive tracking to classify and monitor activities in a site, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, USA, 1998, pp. 22-29.; W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee, Using adaptive tracking to classify and monitor activities in a site, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, USA, 1998, pp. 22-29. |
| [8] | Reid, D. B., An algorithm for tracking multiple targets, IEEE Trans. Automatic Control, 24, 6, 843-854 (1979) |
| [9] | D.P. Huttenlocher, M.E. Leventon, W.J. Rucklidge, Visually-guided navigation by comparing two-dimensional edge images, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 1994, pp. 842-847.; D.P. Huttenlocher, M.E. Leventon, W.J. Rucklidge, Visually-guided navigation by comparing two-dimensional edge images, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 1994, pp. 842-847. |
| [10] | C.F. Olson, Mobile robot self localization by iconic matching of range maps, in: International Conference on Advanced Robotics, 1997, Monterey, CA, USA, pp. 447-452.; C.F. Olson, Mobile robot self localization by iconic matching of range maps, in: International Conference on Advanced Robotics, 1997, Monterey, CA, USA, pp. 447-452. |
| [11] | F. Dellaert, S. Thurn, C. Thorpe, Jacobian images of super-resolved texture maps for model-based motion estimation and tracking, in: Proceedings of the IEEE Workshop Applications of Computer Vision (WACP’98), Princeton, NJ, USA, 1998, 2-7.; F. Dellaert, S. Thurn, C. Thorpe, Jacobian images of super-resolved texture maps for model-based motion estimation and tracking, in: Proceedings of the IEEE Workshop Applications of Computer Vision (WACP’98), Princeton, NJ, USA, 1998, 2-7. |
| [12] | S. Dettmer, A. Seetharamaiah, L. Wang, M. Shah, Model based approach for recognizing human activities from video sequences, in: Univ. Central Florida, Technical Report, 1998.; S. Dettmer, A. Seetharamaiah, L. Wang, M. Shah, Model based approach for recognizing human activities from video sequences, in: Univ. Central Florida, Technical Report, 1998. |
| [13] | L. Goncalves, E. Dibernardo, E. Ursella, P. Perona, Monocular tracking of human arm in 3-d, in: Proceedings of the International Conference on Computer Vision, Cambridge, MA, USA, 1995, 764-770.; L. Goncalves, E. Dibernardo, E. Ursella, P. Perona, Monocular tracking of human arm in 3-d, in: Proceedings of the International Conference on Computer Vision, Cambridge, MA, USA, 1995, 764-770. |
| [14] | D.M. Gavrila, L.S. Davis, 3d model-based tracking of humans in action: a multi-view approach, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 1996, pp. 73-80.; D.M. Gavrila, L.S. Davis, 3d model-based tracking of humans in action: a multi-view approach, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 1996, pp. 73-80. |
| [15] | I. Kakadiaris, D. Metaxas, Model-based estimation of 3-d human motion with occlusion based on active multi-viewpoint selection, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, 1996, pp. 81-87.; I. Kakadiaris, D. Metaxas, Model-based estimation of 3-d human motion with occlusion based on active multi-viewpoint selection, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, 1996, pp. 81-87. |
| [16] | Black, M. J.; Jepson, A., Eigentrackingrobust matching and tracking of articulated objects using a view-based representation, Int. J. Comput. Vision, 26, 1, 63-84 (1988) |
| [17] | C. Bregler, J. Malik, Tracking people with twist and exponential maps, in: Proceedings of the IEEE Computer Society Conference Computer Vision Pattern Recognition, Santa Barbara, CA, USA, 1998, 8-15.; C. Bregler, J. Malik, Tracking people with twist and exponential maps, in: Proceedings of the IEEE Computer Society Conference Computer Vision Pattern Recognition, Santa Barbara, CA, USA, 1998, 8-15. |
| [18] | S.X. Ju, M.J. Black, Y. Yacoob, Cardboard people: a parameterized model of articulated image motion, in: Proceedings of the Second International Conference on Automatic Face and Gesture Recognition, Killington, VT, USA, 1996, 38-44.; S.X. Ju, M.J. Black, Y. Yacoob, Cardboard people: a parameterized model of articulated image motion, in: Proceedings of the Second International Conference on Automatic Face and Gesture Recognition, Killington, VT, USA, 1996, 38-44. |
| [19] | Lowe, D., Robust model-based motion tracking through the integration of search and estimation, Int. J. Comput. Vision, 8, 2, 113-122 (1992) |
| [20] | P. Wunsch, G. Hirzinger, Real-time visual tracking of 3d objects with dynamic handling of occlusion, in: International Conference on Robotics and Automation, Albuquerque, NM, USA, 1997, 2868-2873 (Vol. 4).; P. Wunsch, G. Hirzinger, Real-time visual tracking of 3d objects with dynamic handling of occlusion, in: International Conference on Robotics and Automation, Albuquerque, NM, USA, 1997, 2868-2873 (Vol. 4). |
| [21] | Blackman, S. S.; Tucker R. J. Dempster, G. K.; Roszkowski, S. H., Application of multiple hypothesis tracking to shipboard irst tracking, SPIE, 2759, 441-452 (1996) |
| [22] | Koch, W., Fixed interval retrodiction approach to bayesian imm-mht maneuvering multiple targets, IEEE Trans. Aerospace Electron. Syst., 36, 1, 2-14 (2000) |
| [23] | J.J. Leonard, B.A. Moran, I.J. Cox, M.L. Miller, Underwater sonar data fusion using an efficient multiple hypothesis algorithm, in: IEEE International Conference on Robotics and Automation, Nagaya, Japan, 1995, 2995-3002 (Vol. 3).; J.J. Leonard, B.A. Moran, I.J. Cox, M.L. Miller, Underwater sonar data fusion using an efficient multiple hypothesis algorithm, in: IEEE International Conference on Robotics and Automation, Nagaya, Japan, 1995, 2995-3002 (Vol. 3). |
| [24] | T.J. Cham, J.M. Rehg, A multiple hypothesis approach to figure tracking, Technical Report CRL98/8, Cambridge Research Laboratory Technical Report Series, 1998.; T.J. Cham, J.M. Rehg, A multiple hypothesis approach to figure tracking, Technical Report CRL98/8, Cambridge Research Laboratory Technical Report Series, 1998. |
| [25] | Fortmann, T. E.; Bar-Shalom, Y.; Scheffe, M., Sonar tracking of multiple targets using joint probabilistic data association, J. Ocean Eng., 8, 3, 173-184 (1983) |
| [26] | Smith, P.; Buechler, G., A branching algorithm for discriminating and tracking multiple objects, IEEE Trans. Autom. Control, 20, 101-104 (1975) · Zbl 0298.93019 |
| [27] | Canny, J., A computational approach to edge detection, IEEE Trans. Pattern Anal. Mach. Intell., 8, 6, 679-698 (1986) |
| [28] | A. Etemadi, J.P. Smith, G. Matas, J. Illingworth, J. Kittler, Low level grouping of straight line segments, Technical Report, Department of Electronic and Electrical Engineering, University of Surrey, 1993.; A. Etemadi, J.P. Smith, G. Matas, J. Illingworth, J. Kittler, Low level grouping of straight line segments, Technical Report, Department of Electronic and Electrical Engineering, University of Surrey, 1993. |
| [29] | C.F. Olson, A probabilistic formulation for Hausdorff matching, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, USA, 1998, pp. 150-156.; C.F. Olson, A probabilistic formulation for Hausdorff matching, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Santa Barbara, CA, USA, 1998, pp. 150-156. |
| [30] | Georgia Tech GVU Center/College of Computing. Human Identification at a Distance Database.; Georgia Tech GVU Center/College of Computing. Human Identification at a Distance Database. |
| [31] | Medioni, G.; Cohen, I.; Bremond, F.; Hongeng, S.; Nevatia, R., Event detection and analysis from video streams, IEEE Trans. Pattern Anal. Mach. Intell., 23, 8, 873-889 (2001) |
| [32] | Yi, X.; Camps, O. I., Line-based recognition using a multi-dimensional Hausdorff distance, IEEE Trans. Pattern Anal. Mach. Intell., 21, 9, 901-917 (1999) |
| [33] | Shalom, Y. B.; Fortmann, T. E., Tracking and Data Association (1988), Academic Press: Academic Press New York · Zbl 0634.93001 |
| [34] | T. Kurien, Issues in the Design of Practical Multitarget Tracking Algorithms, in Multitarget-Multisensor Tracking: Advanced Applications. (Ed.), Y. Bar-Shalom, Artech House, Norwood, Massachusetts, December, 1989.; T. Kurien, Issues in the Design of Practical Multitarget Tracking Algorithms, in Multitarget-Multisensor Tracking: Advanced Applications. (Ed.), Y. Bar-Shalom, Artech House, Norwood, Massachusetts, December, 1989. |
| [35] | Huttenlocher, D. P.; Klanderman, G. A.; Rucklidge, W. J., Comparing images using the Hausdorff distance, IEEE Trans. Pattern Anal. Mach. Intell., 15, 9, 850-863 (1993) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
