Objective and subjective evaluation of the influence of small amounts of delay and jitter on a recent first person shooter game

ABSTRACT

There have been several studies in the past years that investigate the impact of network delay on multi-user applications. Primary examples of these applications are real-time multiplayer games. These studies have shown that high network delays and jitter may indeed influence the player's perception of the quality of the game. However, the proposed test values, which are often high, are not always representative for a large percentile of on-line game players. We have therefore investigated the influence of delay and jitter with numbers that are more representative for typical access networks. This in effect allows us to simulate a setup with multiplayer game servers that are located at ISP level and players connected through that ISP's access network. To obtain further true-to-life results, we opted to carry out the test using a recent first person shooter (FPS) game, Unreal Tournament 2003. It can, after all, be expected that this new generation of games has built-in features to diminish the effect of small delay values, given the popularity of playing these games over the Internet. In this paper, we have investigated both subjective perceived quality and objective measurements and will show that both are indeed influenced by even these small delay and jitter values.

REFERENCES

• 1: G. Armitage. Sensitivity of quake3 players to network latency. In ACM SIGCOMM Internet Measurement Workshop 2001, Berkeley, CA, USA, Nov. 2001.
• 2: G. Armitage. An experimental estimation of latency sensitivity in multiplayer quake 3. In 11th IEEE Int. Conf. on Networks (ICON 2003), Sydney, Australia, 2003.
• 3: Y. W. Bernier. Latency compensation methods in client/server in-game protocol design and optimization. In Proc. of Game Developers Conference'01, 2001.
• 4: M. Borella. Source models of network game traffic. In Computer Communications, vol. 23, no. 4, pages 403--410, 2000.
• 5: Epic Games, Atari. Unreal Tournament 2003.
• 6: Tristan Henderson, Latency and User Behaviour on a Multiplayer Game Server, Proceedings of the Third International COST264 Workshop on Networked Group Communication, p.1-13, November 07-09, 2001
• 7: Tristan Henderson , Saleem Bhatti, Networked games: a QoS-sensitive application for QoS-insensitive users?, Proceedings of the ACM SIGCOMM workshop on Revisiting IP QoS: What have we learned, why do we care?, August 25-27, 2003, Karlsruhe, Germany [doi>10.1145/944592.944601]
• 8: Tom Jehaes , Danny De Vleeschauwer , Toon Coppens , Bart Van Doorselaer , Eva Deckers , W. Naudts , K. Spruyt , R. Smets, Access network delay in networked games, Proceedings of the 2nd workshop on Network and system support for games, p.63-71, May 22-23, 2003, Redwood City, California [doi>10.1145/963900.963906]
• 9: National Institute of Standards and Technology. NISTNet. World Wide Web, http://snad.ncsl.nist.gov/itg/nistnet.
• 10: Lothar Pantel , Lars C. Wolf, On the impact of delay on real-time multiplayer games, Proceedings of the 12th international workshop on Network and operating systems support for digital audio and video, May 12-14, 2002, Miami, Florida, USA [doi>10.1145/507670.507674]
• 11: Nathan Sheldon , Eric Girard , Seth Borg , Mark Claypool , Emmanuel Agu, The effect of latency on user performance in Warcraft III, Proceedings of the 2nd workshop on Network and system support for games, p.3-14, May 22-23, 2003, Redwood City, California [doi>10.1145/963900.963901]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.m Miscellaneous

General Terms:
Human Factors, Measurement

Keywords:
analysis, network influences, on-line games