Automatic marking with Sakai

ABSTRACT

Large student numbers often drive teaching staff to consider greater degrees of automation of assessment activities. In introductory Computer Science classes - where submitted programs need to repeatedly be compiled, executed and tested -automation is an obvious route to investigate. This paper reports on an experimental automation system for assessing programming assignments, and its integration with the open source Sakai learning management system. While the system has been an administrative success, feedback from students has identified numerous areas for improvement at the interface of the student and the automatic marker. Furthermore, the use of automation has highlighted the need for teaching software development methodology from an early stage.

REFERENCES

• 1: Christopher Douce , David Livingstone , James Orwell, Automatic test-based assessment of programming: A review, Journal on Educational Resources in Computing (JERIC), v.5 n.3, p.4-es, September 2005 [doi>10.1145/1163405.1163409]
• 2: Lauri Malmi , Ari Korhonen , Riku Saikkonen, Experiences in automatic assessment on mass courses and issues for designing virtual courses, ACM SIGCSE Bulletin, v.34 n.3, September 2002
• 3: Stephen H. Edwards, Rethinking computer science education from a test-first perspective, Companion of the 18th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, October 26-30, 2003, Anaheim, CA, USA [doi>10.1145/949344.949390]
• 4: Sakai Project. 2008. http://www.sakaiproject.org/
• 5: Blackboard. 2008. http://www.blackboard.com/
• 6: IMS Global Learning Consortium. 2008. http://www.imsglobal.org/
• 7: Dougiamas, M. and Taylor, P. C. 2003. Moodle: Using Learning Communities to Create an Open Source Course Management System. In Proceedings of the EDMEDIA 2003 Conference, Honolulu, Hawaii.
• 8: Sakai. 2008. Linktool. https://source.sakaiproject.org/svn/linktool/trunk/linktool.txt
• 9: Facebook. 2008. Developers Documentation. http://developers.facebook.com/documentation.php
• 10: Apache HTTP Server Documentation Project. 2008. Apache suEXEC Support. http://httpd.apache.org/docs/1.3/suexec.html
• 11: Parkinson, J. 2005. Essays marked by computer program. BBC News. http://news.bbc.co.uk/2/hi/uk_news/education/4425423.stm
• 12: Pete Thomas , Kevin Waugh , Neil Smith, Experiments in the automatic marking of ER-diagrams, ACM SIGCSE Bulletin, v.37 n.3, September 2005
• 13: Waugh, K., Thomas, P. and Smith, N. 2007. In Proceedings of the 24th British National Conference on Databases. BNCOD'07. IEEE Computer Society.
• 14: Foubister, F. P., Michaelson, G. J., and Tomes, N. 1997. Automatic assessment of elementary Standard ML programs using Ceilidh. J. Computer Assisted Learning 13 (2). doi:10.1046/j.1365--2729.1997.00012.x
• 15: David Jackson , Michelle Usher, Grading student programs using ASSYST, Proceedings of the twenty-eighth SIGCSE technical symposium on Computer science education, p.335-339, February 27-March 01, 1997, San Jose, California, United States
• 16: Nghi Truong , Paul Roe , Peter Bancroft, Static analysis of students' Java programs, Proceedings of the sixth conference on Australasian computing education, p.317-325, January 01, 2004, Dunedin, New Zealand
• 17: Riku Saikkonen , Lauri Malmi , Ari Korhonen, Fully automatic assessment of programming exercises, ACM SIGCSE Bulletin, v.33 n.3, p.133-136, Sept. 2001
• 18: G. Tremblay , F. Guérin , A. Pons , A. Salah, Oto, a generic and extensible tool for marking programming assignments, Software—Practice & Experience, v.38 n.3, p.307-333, March 2008 [doi>10.1002/spe.v38:3]
• 19: Michael Blumenstein , Steve Green , Ann Nguyen , Vallipuram Muthukkumarasamy, GAME: A Generic Automated Marking Environment for Programming Assessment, Proceedings of the International Conference on Information Technology: Coding and Computing (ITCC'04) Volume 2, p.212, April 05-07, 2004

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.3 COMPUTERS AND EDUCATION
      K.3.1 Computer Uses in Education
          Subjects: Computer-managed instruction (CMI)

Additional Classification:
  K. Computing Milieux
  K.3 COMPUTERS AND EDUCATION
      K.3.2 Computer and Information Science Education
          Subjects: Computer science education; Self-assessment

General Terms:
Economics, Human Factors, Standardization, Verification

Keywords:
Sakai, assessment, automation, interoperability