Performance measures of game controllers in a three-dimensional environment

ABSTRACT

Little work exists on the testing and evaluation of computer-game related input devices. This paper presents five new performance metrics and utilizes two tasks from the literature to quantify differences between input devices in constrained three-dimensional environments, similar to "first-person"-genre games. The metrics are Mean Speed Variance, Mean Acceleration Variance, Percent View Moving, Target Leading Analysis, and Mean Time-to-Reacquire. All measures are continuous, as they evaluate movement during a trial. The tasks involved tracking a moving target for several seconds, with and without target acceleration. An evaluation between an X-Box gamepad and a standard PC mouse demonstrated the ability of the metrics to help reveal and explain performance differences between the devices.

REFERENCES

• 1: Ellson, D. C., The independence of tracking in two and three dimensions with the B-29 pedestal sight. In Report TSEAA-694-2G, 1947, Aero Medical Laboratory.
• 2: Ergonomic requirements for office work with visual display terminals (VDTs) - Part 9 - Requirements for non-keyboard input devices. In ISO ISO/TC 159/SC4/WG3 N147, May 25, 1998. International Organisation for Standardisation.
• 3: Fischer, C. A. and Kondraske, G. V. A new approach to human motion quality measurement. In IEEE/EMBS conference, 1997, pages 1701--1704.
• 4: Fitts, P. M. The information capacity of the human motor system in controlling the amplitude of movement. In Journal of Experimental Psychology, 1954. 47: pages 381--391.
• 5: Gaming goes to Hollywood, in The Economist. 2004.
• 6: Halo 2. 2005, Microsoft Game Studios: http://www.bungie.net/Games/Halo2/.
• 7: Jagacinski, R. J., Repperger, D. W., Ward, S. L. and Moran, M. S. A test of Fitts' law with moving targets, In Human Factors, 1980. 22: pages 225--233.
• 8: Life is Just a Game. The New Atlantis, Winter 2004.
• 9: I. Scott MacKenzie , Tatu Kauppinen , Miika Silfverberg, Accuracy measures for evaluating computer pointing devices, Proceedings of the SIGCHI conference on Human factors in computing systems, p.9-16, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365028]
• 10: Bernard V. Brady, Metroid Prime, BradyGAMES, 2002
• 11: Anant Kartik Mithal , Sarah A. Douglas, Differences in movement microstructure of the mouse and the finger-controlled isometric joystick, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.300-307, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238533]
• 12: Murata, A. An experimental evaluation of mouse, joystick, joycard, lightpen, trackball, and touchscreen for pointing: Basic study on human interface design. In Fourth International Conference on Human-Computer Interaction, 1991, pages 123--127.
• 13: Poulton, E. C. Tracking skill and manual control. 1974, New York: Academic Press.
• 14: XBCD X-Box Windows Device Drivers, http://www.redcl0ud.com/xbcd.html.
• 15: Shumin Zhai , Paul Milgram, Quantifying coordination in multiple DOF movement and its application to evaluating 6 DOF input devices, Proceedings of the SIGCHI conference on Human factors in computing systems, p.320-327, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274689]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Keywords:
computer-game devices, performance evaluation, performance-measurement, target-tracking tasks, video-game devices