Using a pilot study to derive a GUI model for automated testing

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Using a pilot study to derive a GUI model for automated testing

Full text

Pdf (1.22 MB)

Source

ACM Transactions on Software Engineering and Methodology (TOSEM) archive
Volume 18 , Issue 2 (November 2008) table of contents

Article No. 7

Year of Publication: 2008

ISSN:1049-331X

Authors

Qing Xie	Accenture Technology Labs, Chicago, IL
Atif M Memon	University of Maryland, College Park, MD

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 39, Downloads (12 Months): 528, Citation Count: 0

Additional Information:

abstract references index terms collaborative colleagues

Tools and Actions:	Request Permissions Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/1416563.1416567 What is a DOI?

ABSTRACT

Graphical user interfaces (GUIs) are one of the most commonly used parts of today's software. Despite their ubiquity, testing GUIs for functional correctness remains an understudied area. A typical GUI gives many degrees of freedom to an end-user, leading to an enormous input event interaction space that needs to be tested. GUI test designers generate and execute test cases (modeled as sequences of user events) to traverse its parts; targeting a subspace in order to maximize fault detection is a nontrivial task. In this vein, in previous work, we used informal GUI code examination and personal intuition to develop an event-interaction graph (EIG). In this article we empirically derive the EIG model via a pilot study, and the resulting EIG validates our intuition used in previous work; the empirical derivation process also allows for model evolution as our understanding of GUI faults improves. Results of the pilot study show that events interact in complex ways; a GUI's response to an event may vary depending on the context established by preceding events and their execution order. The EIG model helps testers to understand the nature of interactions between GUI events when executed in test cases and why certain events detect faults, so that they can better traverse the event space. New test adequacy criteria are defined for the EIG; new algorithms use these criteria and EIG to systematically generate test cases that are shown to be effective on four fielded open-source applications.

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	J. H. Andrews , L. C. Briand , Y. Labiche, Is mutation an appropriate tool for testing experiments?, Proceedings of the 27th international conference on Software engineering, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062530]
	2	Mikhail Auguston , James Bret Michael , Man-Tak Shing, Environment behavior models for scenario generation and testing automation, Proceedings of the 1st international workshop on Advances in model-based testing, p.1-6, May 15-21, 2005, St. Louis, Missouri
	3	Philip J. Bernhard, A reduced test suite for protocol conformance testing, ACM Transactions on Software Engineering and Methodology (TOSEM), v.3 n.3, p.201-220, July 1994 [doi>10.1145/196092.196088]
	4	T. S. Chow, Testing Software Design Modeled by Finite-State Machines, IEEE Transactions on Software Engineering, v.4 n.3, p.178-187, May 1978 [doi>10.1109/TSE.1978.231496]
	5	Clarke, J. M. 1998. Automated test generation from a behavioral model. In Proceedings of the Pacific Northwest Software Quality Conference (Pnsqc/Pacific Agenda, Portland, OR).
	6	Esmelioglu, S. and Apfelbaum, L. 1997. Automated test generation, execution, and reporting. In Proceedings of the Pacific Northwest Software Quality Conference (Pnsqc/Pacific Agenda, Portland, OR). 127--142.
	7	Mary Jean Harrold , A. Jefferson Offutt , Kanupriya Tewary, An approach to fault modeling and fault seeding using the program dependence graph, Journal of Systems and Software, v.36 n.3, p.273-295, March 1997 [doi>10.1016/S0164-1212(96)00175-6]
	8	Hicinbothom, J. H. and Zachary, W. W. 1993. A tool for automatically generating transcripts of human-computer interaction. In Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting. Special Sessions: Demonstrations, vol. 2. Human Factors and Ergonomics Society, Santa Monica, CA, 1042.
	9	Adele E. Howe , Anneliese von Mayrhauser , Richard T. Mraz, Test Case Generation as an AI Planning Problem, Automated Software Engineering, v.4 n.1, p.77-106, January 1997 [doi>10.1023/A:1008607721339]
	10	Imanian, J. A. 2005. Automatic test case generation for reactive software systems based on environment models. Ph.D. dissertation. Naval Postgraduate School, Monterey, CA.
	11	Paul C. Jorgensen , Carl Erickson, Object-oriented integration testing, Communications of the ACM, v.37 n.9, p.30-38, Sept. 1994 [doi>10.1145/182987.182989]
	12	David J. Kasik , Harry G. George, Toward automatic generation of novice user test scripts, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.244-251, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238519]
	13	Wee Kheng Leow , Siau Cheng Khoo , Yi Sun, Automated Generation of Test Programs from Closed Specifications of Classes and Test Cases, Proceedings of the 26th International Conference on Software Engineering, p.96-105, May 23-28, 2004
	14	Peter M. Maurer, Generating Test Data with Enhanced Context-Free Grammars, IEEE Software, v.7 n.4, p.50-55, July 1990 [doi>10.1109/52.56422]
	15	Atif Memon , Ishan Banerjee , Adithya Nagarajan, GUI Ripping: Reverse Engineering of Graphical User Interfaces for Testing, Proceedings of the 10th Working Conference on Reverse Engineering, p.260, November 13-17, 2003
	16	Memon, A. M. 2001. A comprehensive framework for testing graphical user interfaces. Ph.D. dissertation. Department of Computer Science, University of Pittsburgh, Pittsburgh, PA.
	17	Atif M. Memon, GUI Testing: Pitfalls and Process, Computer, v.35 n.8, p.87-88, August 2002 [doi>10.1109/MC.2002.1023795]
	18	Memon, A. M. 2003. Advances in GUI testing. In Advances in Computers, 58, M. V. Zelkowitz, Ed. Elsevier, Amsterdam, The Netherlands, 150--203.
	19	Atif Memon , Adithya Nagarajan , Qing Xie, Automating regression testing for evolving GUI software, Journal of Software Maintenance: Research and Practice, v.17 n.1, p.27-64, January 2005 [doi>10.1002/smr.305]
	20	Atif M. Memon , Martha E. Pollack , Mary Lou Soffa, Using a goal-driven approach to generate test cases for GUIs, Proceedings of the 21st international conference on Software engineering, p.257-266, May 16-22, 1999, Los Angeles, California, United States [doi>10.1145/302405.302632]
	21	Atif M. Memon , Martha E. Pollack , Mary Lou Soffa, Automated test oracles for GUIs, Proceedings of the 8th ACM SIGSOFT international symposium on Foundations of software engineering: twenty-first century applications, p.30-39, November 06-10, 2000, San Diego, California, United States
	22	Atif M. Memon , Martha E. Pollack , Mary Lou Soffa, Hierarchical GUI Test Case Generation Using Automated Planning, IEEE Transactions on Software Engineering, v.27 n.2, p.144-155, February 2001 [doi>10.1109/32.908959]
	23	Atif M. Memon , Mary Lou Soffa, Regression testing of GUIs, Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering, September 01-05, 2003, Helsinki, Finland
	24	Atif M. Memon , Mary Lou Soffa , Martha E. Pollack, Coverage criteria for GUI testing, Proceedings of the 8th European software engineering conference held jointly with 9th ACM SIGSOFT international symposium on Foundations of software engineering, September 10-14, 2001, Vienna, Austria
	25	Atif M. Memon , Qing Xie, Studying the Fault-Detection Effectiveness of GUI Test Cases for Rapidly Evolving Software, IEEE Transactions on Software Engineering, v.31 n.10, p.884-896, October 2005 [doi>10.1109/TSE.2005.117]
	26	A. Jefferson Offutt , J. Huffman Hayes, A semantic model of program faults, Proceedings of the 1996 ACM SIGSOFT international symposium on Software testing and analysis, p.195-200, January 08-10, 1996, San Diego, California, United States
	27	Sandra Rapps , Elaine J. Weyuker, Data flow analysis techniques for test data selection, Proceedings of the 6th international conference on Software engineering, p.272-278, September 13-16, 1982, Tokyo, Japan
	28	Michael Scheetz , Anneliese von Mayrhauser , Robert France , Eric Dahlman , Adele E. Howe, Generating Test Cases from an OO Model with an AI Planning System, Proceedings of the 10th International Symposium on Software Reliability Engineering, p.250, November 01-04, 1999
	29	Richard K. Shehady , Daniel P. Siewiorek, A Method to Automate User Interface Testing Using Variable Finite State Machines, Proceedings of the 27th International Symposium on Fault-Tolerant Computing (FTCS '97), p.80, June 25-27, 1997
	30	Jaime Spacco , David Hovemeyer , William Pugh, Tracking defect warnings across versions, Proceedings of the 2006 international workshop on Mining software repositories, May 22-23, 2006, Shanghai, China [doi>10.1145/1137983.1138014]
	31	von Mayrhauser, A. and Crawford-Hines, S. 1993. Automated testing support for a robot tape library. In Proceedings of the 4th International Software Reliability Engineering Conference. IEEE Computer Society Press, Los Alamitos, CA, 6--14.
	32	Anneliese von Mayrhauser , Richard T. Mraz , Jeff Walls, Domain Based Regression Testing, Proceedings of the International Conference on Software Maintenance, p.26-35, September 01, 1994
	33	Lee White , Husain Almezen, Generating Test Cases for GUI Responsibilities Using Complete Interaction Sequences, Proceedings of the 11th International Symposium on Software Reliability Engineering (ISSRE'00), p.110, October 08-11, 2000
	34	James Anthony Whittaker, Markov chain techniques for software testing and reliability analysis, University of Tennessee, Knoxville, TN, 1992
	35	James A. Whittaker , Michael G. Thomason, A Markov Chain Model for Statistical Software Testing, IEEE Transactions on Software Engineering, v.20 n.10, p.812-824, October 1994 [doi>10.1109/32.328991]
	36	Denise Woit, Conditional-event usage testing, Proceedings of the 1998 conference of the Centre for Advanced Studies on Collaborative research, p.23, November 30-December 03, 1998, Toronto, Ontario, Canada
	37	Denise M. Woit, Specifying operational profiles for modules, Proceedings of the 1993 ACM SIGSOFT international symposium on Software testing and analysis, p.2-10, June 28-30, 1993, Cambridge, Massachusetts, United States
	38	Qing Xie , Atif M. Memon, Rapid "Crash Testing" for Continuously Evolving GUI-Based Software Applications, Proceedings of the 21st IEEE International Conference on Software Maintenance, p.473-482, September 25-30, 2005 [doi>10.1109/ICSM.2005.72]
	39	Qing Xie , Atif M. Memon, Model-Based Testing of Community-Driven Open-Source GUI Applications, Proceedings of the 22nd IEEE International Conference on Software Maintenance, p.145-154, September 24-27, 2006 [doi>10.1109/ICSM.2006.39]
	40	Xie, T., Marinov, D., Schulte, W., and Notkin, D. 2005. Symstra: A framework for generating object-oriented unit tests using symbolic execution. In Proceedings of the 11th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS 05). Spinger, Berlin, Germany, 365--381.
	41	Andreas Zeller, Isolating cause-effect chains from computer programs, ACM SIGSOFT Software Engineering Notes, v.27 n.6, November 2002 [doi>10.1145/605466.605468]