A daily behavior enabled hidden Markov model for human behavior understanding. (English) Zbl 1140.68458
Summary: This paper presents a Hierarchical Context Hidden Markov Model (HC-HMM) for behavior understanding from video streams in a nursing center. The proposed HC-HMM infers elderly behaviors through three contexts which are spatial, activities, and temporal context. By considering the hierarchical architecture, HC-HMM builds three modules composing the three components, reasoning in the primary and the secondary relationship. The spatial contexts are defined from the spatial structure, so that it is placed as the primary inference contexts. The temporal duration is associated to elderly activities, so activities are placed in the following of spatial contexts and the temporal duration is placed after activities. Between the spatial context reasoning and behavior reasoning of activities, a modified duration HMM is applied to extract activities. According to this design, human behaviors different in spatial contexts would be distinguished in first module. The behaviors different in activities would be determined in second module. The third module is to recognize behaviors involving different temporal duration. By this design, an abnormal signaling process corresponding to different situations is also placed for application. The developed approach has been applied for understanding of elder behaviors in a nursing center. Results have indicated the promise of the approach which can accurately interpret 85% of the elderly behaviors. For abnormal detection, the approach was found to have 90% accuracy, with 0% false alarm.
MSC:
| 68T10 | Pattern recognition, speech recognition |
References:
| [1] | Haritaoglu, I.; Harwood, D.; David, L. S., W4: Real-time surveillance of people and their activities, IEEE Trans. PAMI, 24, 8 (2000) |
| [2] | Ben-Arie, J.; Wang, Z.; Pandit, P.; Rajaram, S., Human activity recognition using multidimensional indexing, IEEE Trans. PAMI, 24, 8 (2002) |
| [3] | Bobick, A. F.; Davis, J. W., The recognition of human movement using temporal templates, IEEE Trans. PAMI, 23, 3 (2001) |
| [4] | R. Polana, R. Nelson, Recognizing activities, in: Proceedings of the International Conference on IAPR, vol. 1, October 1994, pp. 815-818.; R. Polana, R. Nelson, Recognizing activities, in: Proceedings of the International Conference on IAPR, vol. 1, October 1994, pp. 815-818. |
| [5] | H. Fujiyoshi, A.J. Lipton, Real-time human motion analysis by image skeletonization, in: Proceedings of the IEEE Workshop Applications of Computer Vision, October 1998, pp. 15-21.; H. Fujiyoshi, A.J. Lipton, Real-time human motion analysis by image skeletonization, in: Proceedings of the IEEE Workshop Applications of Computer Vision, October 1998, pp. 15-21. |
| [6] | M.M. Rahman, K. Nakamura, S. Ishikawa, Recognizing, human behavior using universal eigenspace, in: Proceedings of the International Conference on Pattern Recognition, vol. 1, August 2002, pp. 295-298.; M.M. Rahman, K. Nakamura, S. Ishikawa, Recognizing, human behavior using universal eigenspace, in: Proceedings of the International Conference on Pattern Recognition, vol. 1, August 2002, pp. 295-298. |
| [7] | N. Robertson, I. Reid, M. Brady, Behaviour recognition and explanation for video surveillance, in: The Institution of Engineering and Technology Conference on IEEE Crime and Security, 2006.; N. Robertson, I. Reid, M. Brady, Behaviour recognition and explanation for video surveillance, in: The Institution of Engineering and Technology Conference on IEEE Crime and Security, 2006. |
| [8] | H. Miyamori, S.-i. Iisaku, Video annotation for content-based retrieval using human behavior analysis and domain knowledge, in: Proceedings of the International Conference on Automatic Face and Gesture Recognition, March 2000.; H. Miyamori, S.-i. Iisaku, Video annotation for content-based retrieval using human behavior analysis and domain knowledge, in: Proceedings of the International Conference on Automatic Face and Gesture Recognition, March 2000. |
| [9] | Zhao, T.; Nevatia, R., Tracking multiple humans in complex situations, IEEE Trans. PAMI, 26, 9 (2004) |
| [10] | N. Carter, D. Young, J. Ferryman, A combined Bayesian Markovian approach for behaviour recognition, in: 18th International Conference on Pattern Recognition, 2006 (ICPR 2006).; N. Carter, D. Young, J. Ferryman, A combined Bayesian Markovian approach for behaviour recognition, in: 18th International Conference on Pattern Recognition, 2006 (ICPR 2006). |
| [11] | P. Kumar, S. Ranganath, H. Weimin, K. Sengupta, Framework for real-time behavior interpretation from traffic video, IEEE Transactions on Intelligent Transportation Systems, March 2005, vol. 6, pp. 43-53.; P. Kumar, S. Ranganath, H. Weimin, K. Sengupta, Framework for real-time behavior interpretation from traffic video, IEEE Transactions on Intelligent Transportation Systems, March 2005, vol. 6, pp. 43-53. |
| [12] | J. Yamato, J. Ohya, K. Ishii, Recognizing human action in time-sequential images using hidden Markov model, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1992 (CVPR ’92), 15-18 June 1992, pp. 379-385.; J. Yamato, J. Ohya, K. Ishii, Recognizing human action in time-sequential images using hidden Markov model, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1992 (CVPR ’92), 15-18 June 1992, pp. 379-385. |
| [13] | Galata, A.; Johnson, N.; Hogg, D., Learning variable-length Markov models of behavior, Comput. Vision Image Understanding, 81, 3, 398-413 (2001) · Zbl 1011.68551 |
| [14] | V. Kellokumpu, M. Pietikäinen, J. Heikkilä, Human activity recognition using sequences of postures, in: Proceedings of the IAPR Conference on Machine Vision Applications (MVA 2005), Tsukuba Science City, Japan, pp. 570-573.; V. Kellokumpu, M. Pietikäinen, J. Heikkilä, Human activity recognition using sequences of postures, in: Proceedings of the IAPR Conference on Machine Vision Applications (MVA 2005), Tsukuba Science City, Japan, pp. 570-573. |
| [15] | Rabiner, L. R., A tutorial on hidden Markov models and selected applications in speech recognition, Proc. IEEE, 77, 2, 257-286 (1989) |
| [16] | M.-Y. Chen, A. Kundu, A complement to variable duration hidden Markov model in handwritten word recognition, in: Proceedings of the IEEE International Conference on Image Processing, 1994 (ICIP-94), vol. 1, 13-16 November 1994, pp. 174-178.; M.-Y. Chen, A. Kundu, A complement to variable duration hidden Markov model in handwritten word recognition, in: Proceedings of the IEEE International Conference on Image Processing, 1994 (ICIP-94), vol. 1, 13-16 November 1994, pp. 174-178. |
| [17] | M. Russell, A segmental HMM for speech pattern modeling, in: IEEE International Conference on Acoustics, Speech, and Signal Processing, 1993 (ICASSP-93), vol. 2, 27-30 April 1993, pp. 499-502.; M. Russell, A segmental HMM for speech pattern modeling, in: IEEE International Conference on Acoustics, Speech, and Signal Processing, 1993 (ICASSP-93), vol. 2, 27-30 April 1993, pp. 499-502. |
| [18] | X. Zhang, J.S. Mason, Improved training using semi-hidden Markov models in speech recognition, in: International Conference on Acoustics, Speech, and Signal Processing, 1989 (ICASSP-89), vol. 1, 23-26 May 1989, pp. 306-309.; X. Zhang, J.S. Mason, Improved training using semi-hidden Markov models in speech recognition, in: International Conference on Acoustics, Speech, and Signal Processing, 1989 (ICASSP-89), vol. 1, 23-26 May 1989, pp. 306-309. |
| [19] | N.T. Nguyen, D.Q. Phung, S. Venkatesh, H. Bui, Learning and detection activities from movement trajectories using the hierarchical hidden Markov model, in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, 2005, pp. 955-960.; N.T. Nguyen, D.Q. Phung, S. Venkatesh, H. Bui, Learning and detection activities from movement trajectories using the hierarchical hidden Markov model, in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, 2005, pp. 955-960. |
| [20] | T.V. Duong, H.H. Bui, D.Q. Phung, S. Venkatesh, Activity recognition and abnormality detection with the switching hidden semi-Markov model, in: IEEE Computer Society Conference on CVPR 2005, vol. 1, pp. 838-845.; T.V. Duong, H.H. Bui, D.Q. Phung, S. Venkatesh, Activity recognition and abnormality detection with the switching hidden semi-Markov model, in: IEEE Computer Society Conference on CVPR 2005, vol. 1, pp. 838-845. |
| [21] | Fine, S.; Singer, Y.; Tishby, N., The hierarchical hidden Markov model analysis and applications, Mach. Learn., 32, 1, 41-62 (1998) · Zbl 0901.68178 |
| [22] | H.H. Bui, D.Q. Phung, S. Venkatesh, Hierarchical hidden Markov models with general state hierarchy, in: Proceedings of the Nineteenth National Conference on Artificial Intelligence, San Jose, CA, 2004, pp. 324-329.; H.H. Bui, D.Q. Phung, S. Venkatesh, Hierarchical hidden Markov models with general state hierarchy, in: Proceedings of the Nineteenth National Conference on Artificial Intelligence, San Jose, CA, 2004, pp. 324-329. |
| [23] | N. Oliver, E. Horvitz, A. Garg, Layered representations for human activity recognition, in: Proceedings of the Fourth IEEE International Conference on Multimodal Interfaces, 14-16 October 2002, pp. 3-8.; N. Oliver, E. Horvitz, A. Garg, Layered representations for human activity recognition, in: Proceedings of the Fourth IEEE International Conference on Multimodal Interfaces, 14-16 October 2002, pp. 3-8. |
| [24] | Bobick, A. F.; Davis, J. W., The recognition of human movement using temporal templates, IEEE Trans. PAMI, 23, 3 (2001) |
| [25] | S. Luhr, S. Venkatesh, G. West, H.H. Bui, Duration abnormality detection in sequence of human activity, Technical Report, Department of Computing, Curtin University of Technology, May 2004.; S. Luhr, S. Venkatesh, G. West, H.H. Bui, Duration abnormality detection in sequence of human activity, Technical Report, Department of Computing, Curtin University of Technology, May 2004. |
| [26] | K. Hara, T. Omori, R. Ueno, Detection of unusual human behavior in intelligent house, in: Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing, 4-6 September 2002, pp. 697-706.; K. Hara, T. Omori, R. Ueno, Detection of unusual human behavior in intelligent house, in: Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing, 4-6 September 2002, pp. 697-706. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
