An algorithm to determine peer-reviewers

ABSTRACT

The peer-review process is the most widely accepted certification mechanism for officially accepting the written results of researchers within the scientific community. An essential component of peer-review is the identification of competent referees to review a submitted manuscript. This article presents an algorithm to automatically determine the most appropriate reviewers for a manuscript by way of a co-authorship network data structure and a relative-rank particle-swarm algorithm. This approach is novel in that it is not limited to a pre-selected set of referees, is computationally efficient, requires no human-intervention, and, in some instances, can automatically identify conflict of interest situations. A useful application of this algorithm would be to open commentary peer-review systems because it provides a weighting for each referee with respects to their expertise in the domain of a manuscript. The algorithm is validated using referee bid data from the 2005 Joint Conference on Digital Libraries.

REFERENCES

• 1: C. Basu, H. Hirsh, W. Cohen, and C. Nevill-Manning. Technical paper recommendation: A study in combining multiple information sources. Journal of Artificial Intelligence Research, 14:231--252, 2001.
• 2: Monica Bianchini , Marco Gori , Franco Scarselli, Inside PageRank, ACM Transactions on Internet Technology (TOIT), v.5 n.1, p.92-128, February 2005 [doi>10.1145/1052934.1052938]
• 3: Johan Bollen , Marko A. Rodriguez , Herbert Van de Sompel , Lyudmila L. Balakireva , Aric Hagberg, The largest scholarly semantic network...ever., Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242789]
• 4: P. Bonacich. Power and centrality: A family of measures. American Journal of Sociology, 92(5):1170--1182, 1987.
• 5: Sergey Brin , Lawrence Page, The anatomy of a large-scale hypertextual Web search engine, Computer Networks and ISDN Systems, v.30 n.1-7, p.107-117, April 1, 1998
• 6: A. Collins and E. Loftus. A spreading activation theory of semantic processing. Psychological Review, 82:407--428, 1975.
• 7: W. J. Conover. Practical nonparametric statistics. John Wiley and Sons, New York, NY, USA, 1971.
• 8: F. Crestani, Application of Spreading Activation Techniques in InformationRetrieval, Artificial Intelligence Review, v.11 n.6, p.453-482, Dec. 1997 [doi>10.1023/A:1006569829653]
• 9: Susan T. Dumais , Jakob Nielsen, Automating the assignment of submitted manuscripts to reviewers, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.233-244, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133205]
• 10: D. Fogaras and B. Rácz. Algorithms and Models for the Web-Graph, chapter Towards Scaling Fully Personalized PageRank, pages 105--117. Springer, 2004.
• 11: J. J. M. Guervós and P. A. C. Valdivieso. Conference paper assignment using a combined greedy/evolutionary algorithm. In Proceedings of the International Conference on Parallel Problem Solving from Nature, pages 602--611, 2004.
• 12: Xiaoming Liu , Johan Bollen , Michael L. Nelson , Herbert Van de Sompel, Co-authorship networks in the digital library research community, Information Processing and Management: an International Journal, v.41 n.6, p.1462-1480, December 2005 [doi>10.1016/j.ipm.2005.03.012]
• 13: M. E. J. Newman. Scientific collaboration networks: I. network construction and fundamental results. Physical Review E, 64(1):016131, 2001.
• 14: M. E. J. Newman. Scientific collaboration networks: Ii. shortest paths, weighted networks, and centrality. Physical Review E, 64(1):016132, 2001.
• 15: M. E. J. Newman. Coauthorship networks and patterns of scientific collaboration. Proceedings of the National Academy of Science, 101:5200--5205, 2004.
• 16: Marko A. Rodriguez, Grammar-based random walkers in semantic networks, Knowledge-Based Systems, v.21 n.7, p.727-739, October, 2008 [doi>10.1016/j.knosys.2008.03.030]
• 17: Marko A. Rodriguez , Johan Bollen , Herbert Van De Sompel, The convergence of digital libraries and the peer-review process, Journal of Information Science, v.32 n.2, p.149-159, April 2006 [doi>10.1177/0165551506062327]
• 18: M. A. Rodriguez, J. Bollen, and H. Van de Sompel. Mapping the bid behavior of conference referees. Journal of Informetrics, 1(1):62--82, 2007.
• 19: T. Sumner. Report on the fifth ACM/IEEE joint conference on digital libraries - cyberinfrastructure for research and education. D-Lib Magazine, 11(7/8), 2005.
• 20: S. Wasserman and K. Faust. Social Network Analysis: Methods and Applications. Cambridge University Press, Cambridge, UK, 1994.
• 21: Scott White , Padhraic Smyth, Algorithms for estimating relative importance in networks, Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, August 24-27, 2003, Washington, D.C. [doi>10.1145/956750.956782]
• 22: D. Yarowsky and R. Florian. Taking the load off the conference chairs: towards a digital paper-routing assistant. In Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in NLP and Very-Large Corpora., 1999.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.7 Digital Libraries

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms

Keywords:
co-authorship networks, peer-review process