Design of a digital library for human movement

ABSTRACT

This paper is focused on a central aspect in the design of our planned digital library for human movement, i.e. on the aspect of representation and recognition of human activity from video data. The method of representation is important since it has a major impact on the design of all the other building blocks of our system such as the user interface/query block or the activity recognition/storage block. In this paper we evaluate a representation method for human movement that is based on sequences of angular poses and angular velocities of the human skeletal joints, for storage and retrieval of human actions in video databases. The choice of a representation method plays an important role in the database structure, search methods, storage efficiency etc.. For this representation, we develop a novel approach for complex human activity recognition by employing multidimensional indexing combined with temporal or sequential correlation. This scheme is then evaluated with respect to its efficiency in storage and retrieval.For the indexing we use postures of humans in videos that are decomposed into a set of multidimensional tuples which represent the poses/velocities of human body parts such as arms, legs and torso. Three novel methods for human activity recognition are theoretically and experimentally compared. The methods require only a few sparsely sampled human postures. We also achieve speed invariant recognition of activities by eliminating the time factor and replacing it with sequence information. The indexing approach also provides robust recognition and an efficient storage/retrieval of all the activities in a small set of hash tables.

REFERENCES

• 1: C. Barron and I. A. Kakadiaris. Estimation anthropometry and pose from a single image. Proc. Conf. Computer Vision and Pattern Recognition, pages 669-676, June 2000.
• 2: C. Bregler , J. Malik, Tracking People with Twists and Exponential Maps, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.8, June 23-25, 1998
• 3: S. K. Chang , Q. Y. Shi , C. W. Yan, Iconic indexing by 2-D strings, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.9 n.3, p.413-428, May 1987 [doi>10.1109/TPAMI.1987.4767923]
• 4: L. Goncalves , E. Di Bernardo , E. Ursella , P. Perona, Monocular tracking of the human arm in 3D, Proceedings of the Fifth International Conference on Computer Vision, p.764, June 20-23, 1995
• 5: Alberto Del Bimbo , Enrico Vicario , Daniele Zingoni, Symbolic Description and Visual Querying of Image Sequences Using Spatio-Temporal Logic, IEEE Transactions on Knowledge and Data Engineering, v.7 n.4, p.609-622, August 1995 [doi>10.1109/69.404033]
• 6: D. Gavrila and L. Davis. Towards 3-d model-based tracking and recognition of human movements. Proc. of the 1995 Int. Workshop on Automatic Face and Gesture Recognition, 1995.
• 7: A. H. Guest. Dance notation: The process of recording movement on paper. London:Dance Books, 1984.
• 8: A. H. Guest. Chore-graphics: A comparison of dance notation systems from the fifteenth century to the present. Systems from the Fifteenth Century to the Present, Gordon and Breach Science Publishers S. A., 1989.
• 9: D. Herbison-Evans. Dance, video, notation and computers. Leonardo, 1988.
• 10: D. Hogg. A program to see a walking person. Image and Vision Computing, 5(20), 1983.
• 11: Shanon X. Ju , Michael J. Black , Yaser Yacoob, Cardboard People: A Parameterized Model of Articulated Image Motion, Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition (FG '96), p.38, October 14-16, 1996
• 12: Erland Jungert, The Observer's Point of View: An Extension of Symbolic Projections, Proceedings of the International Conference GIS - From Space to Territory: Theories and Methods of Spatio-Temporal Reasoning on Theories and Methods of Spatio-Temporal Reasoning in Geographic Space, p.179-195, September 21-23, 1992
• 13: I. A. Kakadiaris , D. Metaxas, Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection, Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition (CVPR '96), p.81, June 18-20, 1996
• 14: T. K. Moon and W. C. Stirling. Mathematical Methods and Algorithms for Signal Processing. Prentice Hall Inc., 2000.
• 15: Richard M. Murray , S. Shankar Sastry , Li Zexiang, A Mathematical Introduction to Robotic Manipulation, CRC Press, Inc., Boca Raton, FL, 1994
• 16: N. Sawasaki , t. Morita , t. Uchiyama, Design and Implementation of High-Speed Visual Tracking System for Real Time Motion Analysis, Proceedings of the International Conference on Pattern Recognition (ICPR '96) Volume III-Volume 7276, p.478, August 25-29, 1996
• 17: J. M. Rehg , T. Kanade, Model-based tracking of self-occluding articulated objects, Proceedings of the Fifth International Conference on Computer Vision, p.612, June 20-23, 1995
• 18: K. Rohr. Incremental recognition of pedestrians from image sequences. Proc. 1995 Comp. Soc. Conference on Computer Vision and Pattern Recognition, pages 8-13, June 1993.
• 19: Robert J. Schilling, Fundamentals of Robotics: Analysis and Control, Simon & Schuster Trade, 1996
• 20: J. Schlenzig, E. Hunter, and R. Jain. Vision based hand gesture interpretation using recursive estimation. Proceedings of the 28th Asilmoar Conference on Signals, Systems and Computers, 1994.
• 21: J. Schwartz, Y. Lamdan, and H. Wolfson. Geometric hashing : A general and efficient model-based recognition scheme. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pages 335-344, 1988.
• 22: A. Prasad Sistla , Ouri Wolfson, Temporal Triggers in Active Databases, IEEE Transactions on Knowledge and Data Engineering, v.7 n.3, p.471-486, June 1995 [doi>10.1109/69.390251]
• 23: T. Starner and A. Pentland. Real time american sign language recognition from video using hidden markov models. Proceedings of the International Symposium on Computer Vision, 1996.
• 24: Andrew D. Wilson , Aaron F. Bobick , Justine Cassell, Temporal Classification of Natural Gesture and Application to Video Coding, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.948, June 17-19, 1997

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION

Additional Classification:
  E. Data

  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Motion; Tracking
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Pattern analysis

General Terms:
Algorithms, Design, Documentation, Experimentation, Human Factors, Management, Measurement, Performance, Theory

Keywords:
human activity recognition, multi dimenional indexing, sequence recognition, temporal correlation