Continuous archival and analysis of user data in virtual and immersive game environments

ABSTRACT

We present a continuous and unobtrusive approach to analyze and reason about users' personal experiences of interacting with virtual and game environments. Focusing on an immersive educational game environment that we are developing, this is achieved through the capture and storage of user's movements and events that occur as a result of interactions with and within immersive environments. Termed immersidata, we then query and analyze immersidata to make sense of user behavior.Two example approaches are described. The first describes an application ISIS (Immersidata analySIS) that provides a tool for analysis of user behavior/experience through the indexing of immersidata with video clips of students' gaming sessions. This approach is described by way of an example to identify the causes of interruptions or breaks in interactions/focus of attention to facilitate the identification of problematic design. In our second example we describe our work towards classifying students' performance through immersidata. To this aim, we describe one example of transforming immersidata into multivariate time series and then by applying feature subset selection techniques we identify the features that differentiate students. We describe the application of this approach to identify novice and expert players with 90\% accuracy. One proposal is to use this to customize the game environment appropriate to the students' ability. Finally, we present future directions for the continuation of the work presented herein and also, the application of the immersidata system to capture, store and analyze personal behavior/experiences and provide appropriate feedback in our work and home environments.

REFERENCES

• 1: Kiyoharu Aizawa , Datchakorn Tancharoen , Shinya Kawasaki , Toshihiko Yamasaki, Efficient retrieval of life log based on context and content, Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences, October 15-15, 2004, New York, New York, USA [doi>10.1145/1026653.1026656]
• 2: T. Marsh, W. L. Wong, E. Carriazo, L. Nocera, K. Yang, A. Varma, H. Yoon, Y. lun huang, C. Kyriakakis, and C. Shahabi. User experiences and lessons learned from developing and implementing an immersive game for the science classroom. In The 11th International Conference on Human-Computer Interaction, Las Vegas, Nevada, USA, July 2005.
• 3: Tim Marsh , Kiyoung Yang , Cyrus Shahabi , Wee Ling Wong , Luciano Nocera , Eduardo Carriazo , Aditya Varma , Hyunjin Yoon , Chris Kyriakakis, Automating the detection of breaks in continuous user experience with computer games, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1056983]
• 4: M. Meehan, B. Insko, M. Whitton, and F. Brooks. Objective measures of presence in virtual environments. In 4th International Workshop on Presence, Philadelphia, PA, USA, 2001.
• 5: Yasuhiko Ooe , Kentaro Yamasaki , Tsukasa Noma, PWS & PHA: posture web server and posture history archiver, Proceedings of the the 1st ACM workshop on Continuous archival and retrieval of personal experiences, October 15-15, 2004, New York, New York, USA [doi>10.1145/1026653.1026669]
• 6: L. R. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257--286, 1989.
• 7: C. Shahabi. AIMS: An immersidata management system. In VLDB CIDR, January 2003.
• 8: C. Shahabi, G. Barish, B. Ellenberger, N. Jiang, M. R. Kolahdouzan, A. Nam, and R. Zimmerman. Immersidata management: Challenges in management of data generated within an immersive environment. In Fifth International Workshop on Multimedia Information Systems (MIS99), 1999.
• 9: M. Slater, A. Brogni, and A. Steed. Physiological responses to breaks in presence: A pilot study. In Proceedings of the 6th International Workshop on Presence, Aalborg, Denmark, 2003.
• 10: M. Slater and A. Steed. A virtual presence counter. Presence: Teleoperators and Virtual Environments, 9(5):413--434, October 2000.
• 11: Jonathan Sykes , Simon Brown, Affective gaming: measuring emotion through the gamepad, CHI '03 extended abstracts on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/765891.765957]
• 12: V. Vapnik. Statistical Learning Theory. New York: Wiley, 1998.
• 13: K. Yang, H. Yoon, and C. Shahabi. A supervised feature subset selection technique for multivariate time series. In International Workshop on Feature Selection for Data Mining: Interfacing Machine Learning with Statistics (FSDM) in conjunction with 2005 SIAM International Conference on Data Mining (SDM'05), Newport Beach, CA, April 2005.
• 14: Hyunjin Yoon , Kiyoung Yang , Cyrus Shahabi, Feature Subset Selection and Feature Ranking for Multivariate Time Series, IEEE Transactions on Knowledge and Data Engineering, v.17 n.9, p.1186-1198, September 2005 [doi>10.1109/TKDE.2005.144]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human information processing

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

General Terms:
Design, Experimentation, Human Factors

Keywords:
breaks, classification, feature subset selection, immersidata analysis, multivariate time series