Single- versus double-blind reviewing: an analysis of the literature

ABSTRACT

The peer review process is generally acknowledged as central to the advancement of scholarly knowledge. It is also vital to the advancement of individual careers.

REFERENCES

• 1: {Abrams 1991} P. A. Abrams, "The Predictive Ability of Peer Review of Grant Proposals: The Case of Ecology and the US National Science Foundation," Social Studies of Science 21(1):111--132, February 1991.
• 2: {Altman et al. 1991} N. Altman, D. Banks, P. Chen, D. Duffy, J. Hardwick, C. Léger, A. Owen, and T. Stukel, "Meeting the Needs of New Statistical Researchers." Statistical Science 6(2):163--174, May 1991.
• 3: {Altman et al. 1992} N. Altman, J. F. Angers, D. Banks, D. Duffy, J. Hardwick, C. Léger, M. Martin, D. Nolan (Chair), A. Owen, D. Politis, K. Roeder, T. N. Stukle and Z. Ying, "Rejoinder," Statistical Science 7(2):265--266, May 1992.
• 4: {Armstrong 1997} J. S. Armstrong, "Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation," Science and Engineering Ethics 3:63--84, 1997.
• 5: {Bachand & Sawallis 2003} R. G. Bachand and P. P. Sawallis, "Accuracy in the Identification of Scholarly and Peer-Reviewed Journals and the Peer-Review Process Across Disciplines," The Serials Librarian 45(2):39--59, 2003.
• 6: {Billard 1993} L. Billard, "Comment," Statistical Science 8(3):320--322. August 1993.
• 7: {Calfee & Valencia 2006} R. C. Calfee and R. R. Valencia, "APA Guide to Preparing Manuscripts for Journal Publication," http://www.apa.org/journals/authors/guide.html, viewed June 21, 2006.
• 8: {Blank 1991} R. M. Blank, "The Effects of Double-Blind versus Single-Blind Reviewing: Experimental Evidence from The American Economic Review," American Economic Review 81(5):1041--1067, December 1991.
• 9: {Borts 1974} G. Borts, "Report of the Managing Editor," American Economic Review 64:476--82, May 1974.
• 10: {Campanario 1998a} J. M. Campanario, "Peer Review for Journals at It Stands Today---Part 1," Science Communication 19(3):181--211, March 1998.
• 11: {Campanario 1998b} J. M. Campanario, "Peer Review for Journals at It Stands Today---Part 2," Science Communication 19(4):277--306, June 1998.
• 12: {Carland et al. 1992} J. A. Carland, J. W. Carland, and C. D. Aby, Jr., "Proposed Codification of Ethicacy in the Publication Process," Journal of Business Ethics 11:95--104, 1992.
• 13: {Ceci & Peters 1984} S. J. Ceci and D. P. Peters, "How blind is blind review?" American Psychologist 39:1491--1494, 1984.
• 14: {Cho et al. 1998} M. K. Cho, A. C. Justice, M. A. Winker, J. A. Berlin, J. F. Waecklerle, M. L. Callaham, and D. Rennie, "Masking author identity in peer review---What factors influence masking success?" Journal of the American Medical Association 280(3):243--245, July 15, 1998.
• 15: {Cox et al. 1993} D. Cox, L. Gleser, M. Perlman, N. Reid, and K. Roeder, "Report of the Ad Hoc Committee of Double-Blind Refereeing," Statistical Science 8(3):310--317, August 1993.
• 16: {Dalton 1995} M. Dalton, "Refereeing of Scholarly Works for Primary Publishing," in Annual Review of Information Science and Technology (ARIST), Volume 30, M. E. Williams (ed.), American Society of Information Science, 1995.
• 17: {Evans et al. 1990} A. T. Evans, R. A. McNutt, R. H. Fletcher, and S. W. Fletcher, "The Effects of Blinding on the Quality of Peer Review: A Randomized Trial," Clinical Research 38(2), 1990.
• 18: {Fisher et al. 1994} M. Fisher, S. B. Friedman, and B. Strauss, "The effects of blinding on acceptance of research papers by peer review," Journal of the American Medical Association 272(2):143--146, July 13, 1994.
• 19: {Fletcher & Fletcher 1997} R. H. Fletcher and S. W. Fletcher, "Evidence for the Effectiveness of Peer Review," Science and Engineering Ethics 3(1):35--50, 1997.
• 20: {Fouad et al. 2000} N. Fouad, S. Brehm, C. I. Hall, M. E. Kite, J. S. Hyde, and N. F. Russo, Women in Academe: Two Steps Forward, One Step Back, Report of the Task Force on Women in Academe, American Psychological Association, 2000. http://www.apa.org/pi/wpo/academe/repthome.html, viewed June 21, 2006.
• 21: {Franzini 1987} L. R. Franzini, "Editors Are Not Blind," American Psychologist, page 104, January 1987.
• 22: {Garfunkel et al. 1994} J. M. Garfunkel, M. H. Ulshen, H. J. Hamrick, and E. E. Lawon, "Effect of Institutional Prestige on Reviewers' Recommendations and Editorial Decisions," Journal of the American Medical Association 272(2):137--138, July 13, 1994.
• 23: {Genest 1993} C. Genest, "Comment," Statistical Science 8(3):323--327, August, 1993.
• 24: {Goldberg 1968} P. Goldberg, "Are some women prejudiced against women?" Transaction 5:28--30, April, 1968.
• 25: {Gordon 1980} M. D. Gordon, "The role of referees in scientific communication," in The Psychology of Written Communication: Selected Readings, J. Hartley (ed.), London, England: Kogan Page, pp. 263--275, 1980.
• 26: {Harnad 1982} S. Harnad, "Peer commentary on peer review," Behavioral and Brain Sciences 5(2):185--186, 1982.
• 27: Shawndra Hill , Foster Provost, The myth of the double-blind review?: author identification using only citations, ACM SIGKDD Explorations Newsletter, v.5 n.2, December 2003 [doi>10.1145/980972.981001]
• 28: {Horrobin 1982} D. F. Horrobin, "Peer review: A philosophically faulty concept which is proving disastrous to science," Behavioral and Brain Sciences 5(2):217--218, 1982.
• 29: {Justice et al. 1998} A. C. Justice, M. K. Cho, M. A. Winker, J. A. Berlin, and D. Rennie, "Does Masking Author Identity Improve Peer Review Quality?" Journal of the American Medical Association 280(3):240--242, July 15, 1998.
• 30: {Laband 1994} D. N. Laband, "A Citation Analysis of the Impact of Blinded Peer Review," Journal of the American Medical Association 272(2):147--149, July 13, 1994.
• 31: {Link 1998} A. M. Link, "US and Non-US Submission: An Analysis of Review Bias," Journal of the American Medical Association 280(3):246--247, July 15, 1998.
• 32: Samuel Madden , David DeWitt, Impact of double-blind reviewing on SIGMOD publication rates, ACM SIGMOD Record, v.35 n.2, p.29-32, June 2006 [doi>10.1145/1147376.1147381]
• 33: {Mahoney et al. 1978} M. J. Mahoney, A. E. Kazdin, and M. Kenigsberg, "Getting Published," Cognitive Therapy and Research 2(1):69--70, 1978.
• 34: {McGiffert 1988} M. McGiffert, "Is Justice Blind? An Inquiry into Peer Review," Scholarly Publishing 20(1):43--48, October 1988.
• 35: {McNutt et al. 1990} R. A. MuNutt, A. T. Evans, R. H. Fletcher, and S. W. Fletcher, "The Effects of Blinding on the Quality of Peer Review," Journal of the American Medical Association 263(10):1371--1376, March 9, 1990.
• 36: {Perlman 1982} D. Perlman, "Reviewer "bias": Do Peters and Ceci protest too much?" Behavioral and Brain Sciences 5(2):231--232, 1982.
• 37: {Peters & Ceci 1982} D. P. Peters and S. J. Ceci, "Peer-Review Practices of Psychological Journals: The Fate of Published Articles, Submitted Again," Behavioral and Brain Sciences 5(2):187--195, 1982.
• 38: {Pitkin 1995} R. M. Pitkin, "Blinded Manuscript Review: An Idea Whose Time Has Come?" Editorial, Obstetrics and Gynecology 85(5 Part 1):781--782, May 1995.
• 39: {Poutney 1996} M. Poutney, "Blinded Reviewing," Editorial, Developmental Medicine and Child Neurology 38(12):1059--1060, December 1996.
• 40: {Rennie 1990} D. Rennie, "Editorial Peer Review in Biomedical Publication: The First International Conference," Journal of the American Medical Association 263(10):1317, March 9, 1990.
• 41: {Rennie & Flanagin 1994} D. Rennie and A. Flanagin, "The Second International Congress on Peer Review in Biomedical Publication," Journal of the American Medical Association 272(2):91, July 13, 1994.
• 42: {Rosenblatt & Kirk 1980} A. Rosenblatt and S. A. Kirk, "Recognition of Authors in Blind Review of Manuscripts," Journal of Social Service Research 3(4):383--394, Summer 1980.
• 43: {Rosenthal 1982} R. Rosenthal, "Reliability and Bias in Peer-Review Practices," Behavioral and Brain Sciences 5(2):235--236, 1991.
• 44: {Smith et al. 2002} J. A. Smith, R. Nixon, A. J. Bueschen, D. D. Venable, and H. H. Henry, "The Impact of Blinded versus Unblinded Abstract Review on Scientific Program Content," Journal of Urology 168(5):2123--2125, November 2002.
• 45: {Snodgrass 2000} R. T. Snodgrass, "Chair's Message," ACM SIGMOD Record 29(1):3, March 2000.
• 46: Richard Snodgrass, Developments at ACM TODS, ACM SIGMOD Record, v.32 n.4, p.14-15, December 2003 [doi>10.1145/959060.959062]
• 47: {Swim et al. 1989} J. K. Swim, E. Borgida, G. Maruyama, and D. G. Myers, "Joan McKay versus John McKay: Do gender stereotypes bias evaluations?" Psychological Bulletin 105(3):409--429, 1989.
• 48: {Tobias & Zibrin 1978} S. Tobias and M. Zibrin, "Does Blind Reviewing Make a Difference?" Educational Researcher 7(1):14--16, January 1978.
• 49: {Tung 2006} A. K. H. Tung, "Impact of Double Blind Reviewing on SIGMOD Publication: A More Detail Analysis," 2 pages, July 2006.
• 50: {van Rooyen 1999} S. van Rooyen, F. Godlee, S. Evans, R. Smith, and N. Black, "Effect of Blinding and Unmasking on the Quality of Peer Review: A Randomized Trial," Journal of General Internal Medicine 14(10):622--624, October 1999.
• 51: {Yankauer 1991} A. Yankauer, "How Blind is Blind Review?" American Journal of Public Health 81(7):843--845, July 1991.
• 52: {Zuckerman & Merton 1971} H. Zuckerman and R. K. Merton, "Patterns of Evaluation in Science: Institutionalization, Structure and Functions of the Referee System," Minerva 9(1):66--100, January 1971.

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.7 THE COMPUTING PROFESSION
      K.7.3 Testing, Certification, and Licensing

Additional Classification:
  A. General Literature
  A.m MISCELLANEOUS

  K. Computing Milieux
  K.7 THE COMPUTING PROFESSION
      K.7.1 Occupations

General Terms:
Design, Documentation, Human Factors