Testing vs. code inspection vs. what else?: male and female end users' debugging strategies

ABSTRACT

Little is known about the strategies end-user programmers use in debugging their programs, and even less is known about gender differences that may exist in these strategies. Without this type of information, designers of end-user programming systems cannot know the "target" at which to aim, if they are to support male and female end-user programmers. We present a study investigating this issue. We asked end-user programmers to debug spreadsheets and to describe their debugging strategies. Using mixed methods, we analyzed their strategies and looked for relationships among participants' strategy choices, gender, and debugging success. Our results indicate that males and females debug in quite different ways, that opportunities for improving support for end-user debugging strategies for both genders are abundant, and that tools currently available to end-user debuggers may be especially deficient in supporting debugging strategies used by females.

REFERENCES

• 1: Robin Abraham , Martin Erwig, UCheck: A spreadsheet type checker for end users, Journal of Visual Languages and Computing, v.18 n.1, p.71-95, February, 2007 [doi>10.1016/j.jvlc.2006.06.001]
• 2: Anonymous, H. M. Customs and Excise Computer Audit Ser-vice, Methodology for the Audit of Spreadsheet Models, 2001. http://customs.hmrc.gov.uk/channelsPortalWebApp/channelsPortalWebApp.portal?_nfpb=true&_pageLabel= pageVAT_ShowContent&id=HMCE_PROD_009443&propertyType=document (downloaded Aug. 28, 2007).
• 3: Victor R. Basili , Richard W. Selby, Comparing the Effectiveness of Software Testing Strategies, IEEE Transactions on Software Engineering, v.13 n.12, p.1278-1296, December 1987 [doi>10.1109/TSE.1987.232881]
• 4: Laura Beckwith , Margaret Burnett , Susan Wiedenbeck , Curtis Cook , Shraddha Sorte , Michelle Hastings, Effectiveness of end-user debugging software features: are there gender issues?, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055094]
• 5: Laura Beckwith , Cory Kissinger , Margaret Burnett , Susan Wiedenbeck , Joseph Lawrance , Alan Blackwell , Curtis Cook, Tinkering and gender in end-user programmers' debugging, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124808]
• 6: Laura Beckwith , Derek Inman , Kyle Rector , Margaret Burnett, On to the Real World: Gender and Self-Efficacy in Excel, Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing, p.119-126, September 23-27, 2007 [doi>10.1109/VLHCC.2007.42]
• 7: Alan F. Blackwell, First Steps in Programming: A Rationale for Attention Investment Models, Proceedings of the IEEE 2002 Symposia on Human Centric Computing Languages and Environments (HCC'02), p.2, September 03-06, 2002
• 8: Margaret Burnett , Curtis Cook , Gregg Rothermel, End-user software engineering, Communications of the ACM, v.47 n.9, September 2004 [doi>10.1145/1015864.1015889]
• 9: Byrnes, J., Miller, C., and Schafer D. Gender differences in risk taking: A meta-analysis. Psych. Bulletin 125 (1999), 367--383.
• 10: Carroll, J. (Ed.), "Minimalism Beyond "The Nurnberg Funnel", MIT Press, Cambridge, MA, 1998.
• 11: Cross, N. Expertise in design: An overview. Design Studies 25, 5 (2004), 427--441.
• 12: Fern, X., Komireddy, C., Burnett, M. Mining interpretable human strategies: A case study, In Proc. ICDM, IEEE (2007).
• 13: Gallagher A., De Lisi R., Holst P., McGillicuddy-De Lisi A., Morely M., Cahalan C. Gender differences in advanced mathematical problem solving, J. Experimental Child Psychology 75, 3 (2000), 165--190.
• 14: Halpern, D. Sex Differences in Cognitive Abilities, 3rd Edition. Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2000.
• 15: Caitlin Kelleher , Randy Pausch , Sara Kiesler, Storytelling alice motivates middle school girls to learn computer programming, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240844]
• 16: Andrew J. Ko , Brad A. Myers, Designing the whyline: a debugging interface for asking questions about program behavior, Proceedings of the SIGCHI conference on Human factors in computing systems, p.151-158, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985712]
• 17: David C. Littman , Jeannine Pinto , Stanley Letovsky , Elliot Soloway, Mental models and software maintenance, Papers presented at the first workshop on empirical studies of programmers on Empirical studies of programmers, p.80-98, June 1986, Washington, D.C., United States
• 18: Meyers-Levy, J. Gender differences in information processing: A selectivity interpretation. In P. Cafferata & A. Tybout, (Eds) Cognitive and Affective Responses to Advertising. Lexington, Ma, Lexington Books, 1989.
• 19: Murthi Nanja , Curtis R. Cook, An analysis of the on-line debugging process, Empirical studies of programmers: second workshop, Ablex Publishing Corp., Norwood, NJ, 1987
• 20: O'Donnell, E. and Johnson, E. The effects of auditor gender and task complexity on information processing efficiency. Int. J. Auditing 5 (2001), 91--105.
• 21: Powell, S., Baker, K., Lawson, B. An Auditing Protocol for Spreadsheet Models, Jan. 2007. http://mba.tuck.dartmouth.edu/
• 22: spreadsheet/product_pubs.html (downloaded Aug. 28, 2007).
• 23: Mary Beth Rosson , Hansa Sinha , Mithu Bhattacharya , Dejin Zhao, Design Planning in End-User Web Development, Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing, p.189-196, September 23-27, 2007 [doi>10.1109/VLHCC.2007.17]
• 24: Neeraja Subrahmaniyan , Cory Kissinger , Kyle Rector , Derek Inman , Jared Kaplan , Laura Beckwith , Margaret Burnett, Explaining Debugging Strategies to End-User Programmers, Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing, p.127-136, September 23-27, 2007 [doi>10.1109/VLHCC.2007.21]
• 25: Earl J. Wagner , Henry Lieberman, Supporting user hypotheses in problem diagnosis, Proceedings of the 9th international conference on Intelligent user interfaces, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964450]
• 26: Mark Weiser, Programmers use slices when debugging, Communications of the ACM, v.25 n.7, p.446-452, July 1982 [doi>10.1145/358557.358577]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

General Terms:
Human Factors

Keywords:
debugging, end-user programming, end-user software engineering, gender, strategy