Open issues in organizing computer systems conferences

ABSTRACT

The Workshop on Organizing Workshops, Conferences, and Symposia for Computer Systems (WOWCS) was organized to "bring together conference organizers (past, present, and future) and other interested people to discuss the issues they confront." In conjunction with WOWCS, we survey some previous publications that discuss open issues related to organizing computer systems conferences, especially concerning conduct and management of the review process. We also list some topics about which we wish WOWCS had received submissions, but did not; these could be good topics for future articles.

REFERENCES

• 1: Mark Allman. A Referee's Plea. http://www.icir.org/mallman/plea.txt, May 2001.
• 2: Mark Allman, Thoughts on reviewing, ACM SIGCOMM Computer Communication Review, v.38 n.2, April 2008 [doi>10.1145/1355734.1355741]
• 3: Tom Anderson and Nick McKeown. Public Reviews of Papers Appearing at HotNets-III. In Proc. Third Workshop on Hot Topics in Networks, Nov. 2004.
• 4: Tom Anderson and Nick McKeown. Program Chairs' Message. In Proc. SIGCOMM, Sep. 2006.
• 5: Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel. Journal Status. Scientometrics, 69(3):669--687, 2006.
• 6: Fabio Casati , Fausto Giunchiglia , Maurizio Marchese, Publish and perish: why the current publication and review model is killing research and wasting your money, Ubiquity, v.8 n.3, p.1-1, 2/23/2007 - 2/30/2007 [doi>10.1145/1226694.1226695]
• 7: Christian Collberg , Stephen Kobourov, Self-plagiarism in computer science, Communications of the ACM, v.48 n.4, p.88-94, April 2005 [doi>10.1145/1053291.1053293]
• 8: NIPS 2006 Program Committee. Nips paper evaluation criteria. http://nips.cc/PaperInformation/EvaluationCriteria, 2006.
• 9: Fred Douglis, Collective Wisdom: A Modest Proposal to Improve Peer Review, Part 1, IEEE Internet Computing, v.11 n.5, p.3-6, September 2007 [doi>10.1109/MIC.2007.103]
• 10: Fred Douglis, Collective Wisdom: A Modest Proposal to Improve Peer Review, Part 1, IEEE Internet Computing, v.11 n.5, p.3-6, September 2007 [doi>10.1109/MIC.2007.103]
• 11: Michalis Faloutsos , Anirban Banerjee , Reza Rejaie, You must be joking: a historic open reviewing at global internet '07, ACM SIGCOMM Computer Communication Review, v.37 n.3, July 2007 [doi>10.1145/1273445.1273457]
• 12: Anja Feldmann, Experiences from the Sigcomm 2005 European shadow PC experiment, ACM SIGCOMM Computer Communication Review, v.35 n.3, July 2005 [doi>10.1145/1070873.1070889]
• 13: George D. Gopen and Judith A. Swan. The Science of Scientific Writing. American Scientist, 78(6):550--558, Nov.-Dec. 1990. Available at http://www.americanscientist.org/template/AssetDetail/assetid/23947.
• 14: Tomáš Grim. A possible role of social activity to explain differences in publication output among ecologists. Oikos, 117(4):484--487, 2008.
• 15: Steve Lawrence, Lee Giles, and Kurt Bollacker. Estimated impact of publication venues in Computer Science. http://citeseer.ist.psu.edu/impact.html.
• 16: Roy Levin and David D. Redell. An Evaluation of the Ninth SOSP Submissions; or, How (and How Not) to Write a Good Systems Paper. ACM SIGOPS Operating Systems Review, 17(3):35--40, July 1983. Available at http://www.usenix.org/events/samples/submit/advice.html.
• 17: Samuel Madden , David DeWitt, Impact of double-blind reviewing on SIGMOD publication rates, ACM SIGMOD Record, v.35 n.2, p.29-32, June 2006 [doi>10.1145/1147376.1147381]
• 18: Jeffrey C. Mogul. Conference Report on WOWCS '08. ;login:, 33(4), August 2008.
• 19: National Science and Technology Council, Committee on Fundamental Science, Subcommittee on Research. Assessing fundamental science. http://www.nsf.gov/statistics/ostp/assess/, July 1996.
• 20: ACM/IEEE Health of Conferences Committee. http://wiki.acm.org/healthcc, 2006.
• 21: Konstantina Papagiannaki, Author feedback experiment at PAM 2007, ACM SIGCOMM Computer Communication Review, v.37 n.3, July 2007 [doi>10.1145/1273445.1273456]
• 22: Ian Parberry, A guide for new referees in theoretical computer science, Information and Computation, v.112 n.1, p.96-116, July 1994 [doi>10.1006/inco.1994.1053]
• 23: Timothy Roscoe. Writing reviews for systems conferences. http://people.inf.ethz.ch/troscoe/pubs/reviewwriting.pdf, 2007.
• 24: Pamela Samuelson, Self-plagiarism or fair use, Communications of the ACM, v.37 n.8, p.21-25, Aug. 1994 [doi>10.1145/179606.179731]
• 25: Henning Schulzrinne. EDAS: Editor's Assistant. http://edas.info/doc/.
• 26: SIGCOMM. SIGCOMM Frequently Asked Questions (FAQ). http://www.sigcomm.org/faq.html.
• 27: SIGCOMM Technical Advisory Committee. Improving Sigcomm: A few straw proposals. http://www.sigcomm.org/admin/July2001RepFinal.pdf, 2001.
• 28: Alan Jay Smith, The Task of the Referee, Computer, v.23 n.4, p.65-71, April 1990 [doi>10.1109/2.55470]
• 29: Richard Snodgrass, Single- versus double-blind reviewing: an analysis of the literature, ACM SIGMOD Record, v.35 n.3, p.8-21, September 2006 [doi>10.1145/1168092.1168094]
• 30: Rick Snodgrass. Summary of Conference Management Software. http://www.acm.org/sigs/sgb/summary.html, 1999.
• 31: Anthony K. H. Tung, Impact of double blind reviewing on SIGMOD publication: a more detail analysis, ACM SIGMOD Record, v.35 n.3, p.6-7, September 2006 [doi>10.1145/1168092.1168093]
• 32: Daniel S. Weld. Personal communication. 2008.
• 33: David Wetherall and Tom Anderson. Personal communication (unpublished manuscript). 2008.
• 34: Wikipedia. Impact factor. http://en.wikipedia.org/wiki/Impact_factor.

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.7 THE COMPUTING PROFESSION
      K.7.m Miscellaneous

Keywords:
conferences, program committees, reviewing, workshops