Usability heuristics for networked multiplayer games

ABSTRACT

Networked multiplayer games must support a much wider variety of interactions than single-player games because networked games involve communication and coordination between players. This means that designers must consider additional usability issues that relate to group play - but there are currently no usability engineering methods that are specifically oriented towards the needs of multiplayer games. To address this problem, we developed a new set of usability heuristics, called Networked Game Heuristics (NGH), which can be used in the design and evaluation of networked multiplayer games. The new heuristics were identified by analyzing problem reports from 382 reviews of networked PC games, covering six main genres. We aggregated problem reports into ten problem categories (covering issues from session management to cheating to training for novice players) and developed heuristics that describe how these usability problems can be avoided. We tested the new heuristics by having evaluators use them and an existing set to assess the usability of two networked games. Evaluators found more usability problems with NGH, and stated that the new heuristics were better for evaluating multiplayer game usability. Our research is the first to present networked game heuristics that are derived from real problem reports, and the first to evaluate the heuristics' effectiveness in a realistic usability test.

REFERENCES

• 1: Abner, W. Review of Heroes of Might and Magic V, pc.gamespy.com/pc/heroes-of-might-magic-5/710168p3.html, 2006.
• 2: Accardo, S. Review of Medal of Honor: Airborne, pc.gamespy.com/pc/medal-of-honor-airborne/820348p1.html, 2007.
• 3: Thor Alexander, Massively Multiplayer Game Developemnt, Charles River Media, Inc., Rockland, MA, 2003
• 4: Annett, J. and Duncan, K.D. Task analysis and training design. Occupational Psychology 12, 1967, 211--221.
• 5: Kevin Baker , Saul Greenberg , Carl Gutwin, Empirical development of a heuristic evaluation methodology for shared workspace groupware, Proceedings of the 2002 ACM conference on Computer supported cooperative work, November 16-20, 2002, New Orleans, Louisiana, USA [doi>10.1145/587078.587093]
• 6: Steve Cornett, The usability of massively multiplayer online roleplaying games: designing for new users, Proceedings of the SIGCHI conference on Human factors in computing systems, p.703-710, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985781]
• 7: Heather Desurvire , Martin Caplan , Jozsef A. Toth, Using heuristics to evaluate the playability of games, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986102]
• 8: Diot, C., Gautie, L. A Distributed Architecture for Multiplayer Interactive Applications on the Internet. IEEE Network, 13(4), 6--15, 1999.
• 9: Ducheneaut, N., Moore, R. J., & Nickell, E. Designing for sociability in Massively Multiplayer Games: An examination of the "third places" of SWG. Proc. of the Other Players Conference, 2004.
• 10: Dykstra, D.J. A Comparison of Heuristic Evaluation and Usability Testing: The Efficacy of a Domain-Specific Heuristic Checklist. Ph.D. dissertation, Department of Industrial Engineering, Texas A&M University, 1993.
• 11: Federoff, M. Heuristics and Usability Guidelines for the Creation and Evaluation of Fun in Video Games. Indiana University Master of Science Thesis, 2002.
• 12: Tracy Fullerton , Christopher Swain , Steven Hoffman, Game Design Workshop: Designing, Prototyping, and Playtesting Games, CMP Books, 2004
• 13: Gamespot staff. Review of Tom Clancy's Rainbow Six Vegas, www.gamespot.com/pc/action/rainbowsixvegas/review.html?sid=6163307, 2006.
• 14: GameSpy staff. Review of Dungeons & Dragons Online: Stormreach, pc.gamespy.com/pc/dungeons-dragons-online/696856p4.html, 2006.
• 15: Carl Gutwin , Saul Greenberg, The Mechanics of Collaboration: Developing Low Cost Usability Evaluation Methods for Shared Workspaces, Proceedings of the 9th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, p.98-103, June 04-16, 2000
• 16: Carl Gutwin , Saul Greenberg, The effects of workspace awareness support on the usability of real-time distributed groupware, ACM Transactions on Computer-Human Interaction (TOCHI), v.6 n.3, p.243-281, Sept. 1999 [doi>10.1145/329693.329696]
• 17: Jacob Habgood , Mark Overmars, The Game Maker's Apprentice: Game Development for Beginners, Apress, Berkely, CA, 2006
• 18: Harker, C. Review of RYL: Path of the Emperor, pc.gamespy.com/pc/risk-your-life-2-path-of-the-emperor/635953p1.html, 2005.
• 19: Harms, W. Review of The Lord of the Rings: The Battle for Middle-earth II, pc.gamespy.com/pc/the-lord-of-the-rings-the-battle-for-middle-earth-2/693176p3, 2006.
• 20: Harms, W. Review of Rainbow Six: Lockdown, pc.gamespy.com/pc/rainbow-six-4-tentative-title/ 690399p2.html, 2006.
• 21: Koivisto, E.M. Supporting Communities in Massively Multiplayer Online Role-Playing Games by Game Design. Proc. Level Up, 2003.
• 22: Hannu Korhonen , Elina M. I. Koivisto, Playability heuristics for mobile multi-player games, Proceedings of the 2nd international conference on Digital interactive media in entertainment and arts, September 19-21, 2007, Perth, Australia [doi>10.1145/1306813.1306828]
• 23: Hannu Korhonen , Elina M. I. Koivisto, Playability heuristics for mobile games, Proceedings of the 8th conference on Human-computer interaction with mobile devices and services, September 12-15, 2006, Helsinki, Finland [doi>10.1145/1152215.1152218]
• 24: Kosak, D. Review of Crysis, pc.gamespy.com/pc/ea-crytek-title-untitled-project/839785p1.html, 2007.
• 25: Kosak, D. Review of Age of Pirates: Caribbean Tales, pc.gamespy.com/pc/age-of-pirates/737252p2, 2006.
• 26: Kosak, D. and Lopez, M. Review of Guild Wars, pc.gamespy.com/pc/guild-wars/613203p1.html, 2005.
• 27: Kuo, L.C. Review of Trackmania United, pc.gamespy.com/ pc/trackmania-united/799496p1.html, 2007.
• 28: Kuo, L.C. Review of GTR 2, pc.gamespy.com/pc/gtr-2/751848p1.html, 2006.
• 29: Kuo, L.C. Review of Tom Clancy's Splinter Cell Double Agent, pc.gamespy.com/pc/tom-clancys-splinter-cell-4/751217p1.html, 2006.
• 30: Madigan, J. Review of Cold War Conflicts: Days in the Field 1950-1973, http://pc.gamespy.com/pc/cold-war-conflict/626650p1.html, 2005.
• 31: Jakob Nielsen , Robert L. Mack, Usability inspection methods, John Wiley & Sons, Inc., New York, NY, 1994
• 32: Nielsen, J. How to Conduct a Heuristic Evaluation. www.useit.com/papers/heuristic/, 2007.
• 33: Nielsen, J. Severity Ratings for Usability Problems. www.useit.com/papers/heuristic/severityrating, 2007.
• 34: Osborne, S. Review of TrackMania Sunrise, pc.gamespy.com/pc/trackmania-sunrise/620071p1, 2005.
• 35: David Pinelle , Nelson Wong , Tadeusz Stach, Heuristic evaluation for games: usability principles for video game design, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357282]
• 36: David Pinelle , Carl Gutwin, Groupware walkthrough: adding context to groupware usability evaluation, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503458]
• 37: David Pinelle , Carl Gutwin , Saul Greenberg, Task analysis for groupware usability evaluation: Modeling shared-workspace tasks with the mechanics of collaboration, ACM Transactions on Computer-Human Interaction (TOCHI), v.10 n.4, p.281-311, December 2003 [doi>10.1145/966930.966932]
• 38: Rausch, A. Review of Hellgate London, pc.gamespy.com/pc/hellgate-london/832252p3.html, 2007.
• 39: Smed, ,J., Kaukoranta, T., and Hakonen, H. Aspects of Networking in Multiplayer Computer Games. The Electronic Library, 20(2):87--97, 2002.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

General Terms:
Design, Human Factors

Keywords:
NGH, game usability, heuristic evaluation, multiplayer games, networked game heuristics, networked games, usability