Monitoring visual focus of attention via local discriminant projection

ABSTRACT

In this paper, we present a system to monitor a subject's Visual Focus of Attention (VFOA) based on his/her head poses. The system first detects faces from video images and determines if the detected face is a frontal or profile face. If a frontal face is detected, the system further estimates the head pose from the face image. Instead of estimating accurate head poses through detection or tracking methods, we formulate the problem as a classification problem and classify the head pose into one of a predefined number of poses using a local discriminant projection (LDP) method. The LDP method uses two graphs for the modeling the head pose embedding, one is the nearest native neighbor graph, the other is the nearest invader graph. We evaluate the LDP method in CAS-PEAL Database with 21 head poses and a realistic data set with 9 poses collected from our application scenario. The experimental results indicate that our approach outperforms other methods. We describe the implementation of the system with an application in monitoring customers' VFOA in a display window that exhibits merchandise in a shop. The system can be used to index and retrieve information for customer analysis.

REFERENCES

• 1: Mikhail Belkin , Partha Niyogi, Laplacian Eigenmaps for dimensionality reduction and data representation, Neural Computation, v.15 n.6, p.1373-1396, June 2003 [doi>10.1162/089976603321780317]
• 2: B. Brumitt, J. Krumm, B. Meyers, and S. Shafer. Let there be light: Comparing interfaces for homes of the future. IEEE Personal Communications, 2000.
• 3: Longbin Chen , Lei Zhang , Yuxiao Hu , Mingjing Li , Hongjiang Zhang, Head Pose Estimation using Fisher Manifold Learning, Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures, p.203, October 17-17, 2003
• 4: Yun Fu , Thomas S. Huang, Graph Embedded Analysis for Head Pose Estimation, Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition, p.3-8, April 10-12, 2006 [doi>10.1109/FGR.2006.60]
• 5: W. Gao, B. Cao, and S. S. et.al. The cas-peal large-scale chinese face database and baseline evaluations. IEEE Trans. System, Man and Cybernetics-Part A: Systems and Humans, 38(1):149--161, 2008.
• 6: Xiaofei He , Shuicheng Yan , Yuxiao Hu , Partha Niyogi , Hong-Jiang Zhang, Face Recognition Using Laplacianfaces, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.3, p.328-340, March 2005 [doi>10.1109/TPAMI.2005.55]
• 7: Jiani Hu , Weihong Deng , Jun Guo , Weiran Xu, Locality discriminating indexing for document classification, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277860]
• 8: Y. Hu, L. Chen, Y. Zhou, and H. Zhang. Estimating face pose by facial asymmetry and geometry. Proceeding of IEEE International Conference Automatic Face and Gesture Recogntion, 2006.
• 9: S. Langton. The mutual influence of gaze and head orientation in the analysis of social attention direction. The Quaterly Journal of Experimental Psychology, 53.
• 10: S. Li, Q.-D. Fu, L. Gu, B. Scholkopf, Y. Cheng, and H. Zhang. Kernel machine based learning for multi-view face detection and pose estimation. Proceeding of IEEE International Conference on Computer Vision, 2001.
• 11: Z. Li, Y. Fu, J. Yuan, T. Huang, and Y. Wu. Query driven localized linear discriminant models for head pose estimation. Proceeding of IEEE International Conference on Multimedia & Expo, 2007.
• 12: Paul P. Maglio , Teenie Matlock , Christopher S. Campbell , Shumin Zhai , Barton A. Smith, Gaze and Speech in Attentive User Interfaces, Proceedings of the Third International Conference on Advances in Multimodal Interfaces, p.1-7, October 14-16, 2000
• 13: A. Pece. From cluster tracking to people counting. Proceeding of 3rd PETS Workshop, 2002.
• 14: Bisser Raytchev , Ikushi Yoda , Katsuhiko Sakaue, Head Pose Estimation by Nonlinear Manifold Learning, Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 4, p.462-466, August 23-26, 2004 [doi>10.1109/ICPR.2004.432]
• 15: S. Roweis and L. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290:2323--2326, 2000.
• 16: Kevin Smith , Sileye O. Ba , Daniel Gatica-Perez , Jean-Marc Odobez, Tracking the multi person wandering visual focus of attention, Proceedings of the 8th international conference on Multimodal interfaces, November 02-04, 2006, Banff, Alberta, Canada [doi>10.1145/1180995.1181048]
• 17: R. Stiefelhagen , J. Yang, Gaze Tracking for Multimodal Human-Computer Interaction, Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97) -Volume 4, p.2617, April 21-24, 1997
• 18: J. Tenenbaum, V. de Silva, and J. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290:2319--2323, 2000.
• 19: Paul Viola , Michael J. Jones, Robust Real-Time Face Detection, International Journal of Computer Vision, v.57 n.2, p.137-154, May 2004 [doi>10.1023/B:VISI.0000013087.49260.fb]
• 20: A. Yarbus. Eye movements and vision. New York Plenum press, 1967.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.9 Applications

General Terms:
Algorithms

Keywords:
attention information retrieval, eye localization, face detection, head pose estimation, local discriminant projection, visual focus of attention