Writing good software engineering research papers: minitutorial

ABSTRACT

Software engineering researchers solve problems of several different kinds. To do so, they produce several different kinds of results, and they should develop appropriate evidence to validate these results. They often report their research in conference papers. I analyzed the abstracts of research papers submitted to ICSE 2002 in order to identify the types of research reported in the submitted and accepted papers, and I observed the program committee discussions about which papers to accept. This report presents the research paradigms of the papers, common concerns of the program committee, and statistics on success rates. This information should help researchers design better research projects and write papers that present their results to best advantage.

REFERENCES

• 1: Victor R. Basili, The Experimental Paradigm in Software Engineering, Proceedings of the International Workshop on Experimental Software Engineering Issues: Critical Assessment and Future Directions, p.3-12, September 14-18, 1992
• 2: Geoffery C. Bowker , Susan Leigh Star, Sorting things out: classification and its consequences, MIT Press, Cambridge, MA, 2000
• 3: F. P. Brooks, Grasping reality through illusion—interactive graphics serving science, Proceedings of the SIGCHI conference on Human factors in computing systems, p.1-11, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57168]
• 4: Rebecca Bumett. Technical Communication. Thomson Heinle 2001.
• 5: Thomas F. Gieryn. Cultural Boundaries of Science: Credibility on the line. Univ of Chicago Press, 1999.
• 6: ICSE 2002 Program Committee. Types of ICSE papers. http://icse-conferences.org/2002/info/paperTypes.html
• 7: Leon Osterweil, Impact project panel: “determining the impact of software engineering research upon practice”, Proceedings of the 23rd International Conference on Software Engineering, p.697, May 12-19, 2001, Toronto, Ontario, Canada
• 8: Ellen Isaacs and John Tang. Why don't more non-North-American papers get accepted to CHI? http://acm.org/sigchi/bulletin/1996.1/isaacs.html
• 9: Ralph E. Johnson , Kent Beck , Grady Booch , William Cook , Richard Gabriel , Rebecca Wirfs-Brock, How to get a paper accepted at OOPSLA (panel), Proceedings of the eighth annual conference on Object-oriented programming systems, languages, and applications, p.429-436, September 26-October 01, 1993, Washington, D.C., United States
• 10: Jim Kajiya. How to Get Your SIGGRAPH Paper Rejected. Mirrored at http://www.cc.gatech.edu/student.services/phd/phd-advice/kajiya
• 11: Roy Levin and David D. Redell. How (and How Not) to Write a Good Systems Paper. ACM SIGOPS Operating Systems Review, Vol. 17, No. 3 (July, 1983), pages 35--40. http://fip.digital.com/pub/DEC/SRC/other/SOSPadvice.txt
• 12: William Newman, A preliminary analysis of the products of HCI research, using pro forma abstracts, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.278-284, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191766]
• 13: William Newman et al. Guide to Successful Papers Submission at CHI2001. http://acm.org/sigs/sigchi/chi2001/call/submissions/guide-papers.html
• 14: Alan Snyder, How to get your paper accepted at OOPSLA, Conference proceedings on Object-oriented programming systems, languages, and applications, p.359-363, October 06-11, 1991, Phoenix, Arizona, United States
• 15: Craig Partridge. How to Increase the Chances your Paper is Accepted at ACM SIGCOMM. http://www.acm.org/sigcomm/conference-misc/author-guide.html
• 16: William Pugh and PDLI 1991 Program Committee. Advice to Authors of Extended Abstracts. http://acm.org/sigsoft/conferences/pughadvice.html
• 17: Samuel Redwine, et al. DoD Related Software Technology Requirements, Practices, and Prospects for the Future. IDA Paper P-1788, June 1984.
• 18: Samuel T. Redwine, Jr. , William E. Riddle, Software technology maturation, Proceedings of the 8th international conference on Software engineering, p.189-200, August 28-30, 1985, London, England
• 19: Mary Shaw, The coming-of-age of software architecture research, Proceedings of the 23rd International Conference on Software Engineering, p.656, May 12-19, 2001, Toronto, Ontario, Canada
• 20: Mary Shaw. What makes good research in software engineering? Presented at ETAPS 02, appeared in Opinion Corner department, Int'l Jour on Software Tools for Tech Transfer, vol 4, DOI 10.1007/s10009-002-0083-4, June 2002.
• 21: SigGraph 2003 Call for Papers. http://www.siggraph.org/s2003/cfp/papers/index.html
• 22: Walter F. Tichy , Paul Lukowicz , Lutz Prechelt , Ernst A. Heinz, Experimental evaluation in computer science: a quantitative study, Journal of Systems and Software, v.28 n.1, p.9-18, Jan. 1995 [doi>10.1016/0164-1212(94)00111-Y]
• 23: Walter F. Tichy, Should Computer Scientists Experiment More?, Computer, v.31 n.5, p.32-40, May 1998 [doi>10.1109/2.675631]
• 24: Marvin V. Zelkowitz and Delores Wallace. Experimental validation in software engineering. Information and Software Technology, Vol 39, no 11, 1997, pp. 735--744.
• 25: Marvin V. Zelkowitz , Dolores R. Wallace, Experimental Models for Validating Technology, Computer, v.31 n.5, p.23-31, May 1998 [doi>10.1109/2.675630]
• 26: Mary-Claire van Leunen and Richard Lipton. How to have your abstract rejected. http://acm.org/sigsoft/conferences/vanLeunenLipton.html

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.7 THE COMPUTING PROFESSION
      K.7.m Miscellaneous

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.0 General

General Terms:
Algorithms, Performance

Keywords:
research design, research paradigms, software profession, technical writing, validation