Measuring the performance of interactive applications with listener latency profiling

ABSTRACT

When Java developers need to improve the performance of their applications, they usually use one of the many existing profilers for Java. These profilers generally capture a profile that represents the execution time spent in each method. The developer can thus focus her optimization efforts on the methods that consume the most time. In this paper we argue that this type of profile is insufficient for tuning interactive applications. Interactive applications respond to user events, such as mouse clicks and key presses. The perceived performance of interactive applications is directly related to the response time of the program.

In this paper we present listener latency profiling, a profiling approach with two distinctive characteristics. First, we call it latency profiling because it helps developers to find long latency operations. Second, we call it listener profiling because it abstracts away from method-level profiles to compute the latency of the various listeners. This allows a developer to reason about performance with respect to listeners, also called observers, the high level abstraction at the core of any interactive Java application.

We present our listener latency profiling approach, describe LiLa, our implementation, validate it on a set of micro-benchmarks, and evaluate it on a complex real-world interactive application.

REFERENCES

• 1: I. Ceaparu, J. Lazar, K. Bessiere, J. Robinson, and B. Shneiderman. Determining causes and severity of end-user frustration. International Journal of Human-Computer Interaction, 17(3), 2004.
• 2: James R. Dabrowski , Ethan V. Munson, Is 100 Milliseconds Too Fast?, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634255]
• 3: Yasuhiro Endo , Margo Seltzer, Improving interactive performance using TIPME, Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.240-251, June 18-21, 2000, Santa Clara, California, United States
• 4: Yasuhiro Endo , Zheng Wang , J. Bradley Chen , Margo Seltzer, Using latency to evaluate interactive system performance, Proceedings of the second USENIX symposium on Operating systems design and implementation, p.185-199, October 29-November 01, 1996, Seattle, Washington, United States
• 5: Kristián Flautner , Rich Uhlig , Steve Reinhardt , Trevor Mudge, Thread-level parallelism and interactive performance of desktop applications, Proceedings of the ninth international conference on Architectural support for programming languages and operating systems, p.129-138, November 2000, Cambridge, Massachusetts, United States
• 6: Eclipse Foundation. The PerformanceStats API: Gathering performance statistics in Eclipse. http://www.eclipse.org/eclipse/platform-core/documents/3.1/perf_stats.html, 2005.
• 7: Eclipse Foundation. Eclipse Project. http://www.eclipse.org/eclipse/, 2006.
• 8: I. Scott MacKenzie , Colin Ware, Lag as a determinant of human performance in interactive systems, Proceedings of the INTERCHI '93 conference on Human factors in computing systems, p.488-493, May 1993, Amsterdam, The Netherlands
• 9: Jeff McAffer , Jean-Michel Lemieux, Eclipse Rich Client Platform: Designing, Coding, and Packaging Java(TM) Applications, Addison-Wesley Professional, 2005
• 10: Steve Northover , Mike Wilson, Swt: the standard widget toolkit, volume 1, Addison-Wesley Professional, 2004
• 11: ObjectWeb. ASM. Web pages at http://asm.objectweb.org/.
• 12: Tristan Richardson , Quentin Stafford-Fraser , Kenneth R. Wood , Andy Hopper, Virtual Network Computing, IEEE Internet Computing, v.2 n.1, p.33-38, January 1998 [doi>10.1109/4236.656066]
• 13: Ben Shneiderman, Response time and display rate in human performance with computers, ACM Computing Surveys (CSUR), v.16 n.3, p.265-285, Sept. 1984 [doi>10.1145/2514.2517]
• 14: Ben Shneiderman, Designing the user interface: strategies for effective human-computer interaction, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1986
• 15: Kathy Walrath , Mary Campione , Alison Huml , Sharon Zakhour, The JFC Swing Tutorial: A Guide to Constructing GUIs, Second Edition, Addison Wesley Longman Publishing Co., Inc., Redwood City, CA, 2004
• 16: Nickolai Zeldovich , Ramesh Chandra, Interactive performance measurement with VNCplay, Proceedings of the annual conference on USENIX Annual Technical Conference, p.54-54, April 10-15, 2005, Anaheim, CA

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Performance measures

General Terms:
Human Factors, Measurement, Performance

Keywords:
GUI, latency, listeners, profiling