Trust and the automated assessment of IT skills

ABSTRACT

This paper discusses the problems of introducing automated assessment to an already existing set of IT skills examinations marked by human examiners, concentrating on issues of trust. The paper considers the issues which influence trust in traditional assessment, how the same issues differ for computer-based assessment and how the influence of the differences on trust can be minimised. Finally the paper discusses how future developments may be influenced by issues of trust.

REFERENCES

• 1: Blayney, P., Use of a Spreadsheet Based Marking System for Assisted Learning and Assessment, Proceedings of the 12th Annual Conference of ASCILITE (December 1995), 34--40.
• 2: Bloom. B.S. ed Taxonomy of Educational Objectives Handbook I: Cognitive Domain, McKay, New York, 1956.
• 3: Stuart K. Card , Allen Newell , Thomas P. Moran, The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1983
• 4: CLAIT Tutor's Handbook and Syllabus, 3rd Edition, L706, OCR, Coventry, October 1998.
• 5: Dowsing, R.D., Long, S and Craven, P., An Analysis of the difference between traditional and computer-based assessment of IT-skills in Proc ASCILITE 2000 (Coffs Harbour, December,2000), 477--486.
• 6: R. D. Dowsing , S. Long , M. R. Sleep, Assessing word processing skills by computer, Information Services and Use, v.18 n.1-2, p.15-24, Sept. 1998
• 7: ECDL, European Computer Driving Licence, http://www.ecdl.co.uk, 2000.
• 8: Edmonds, T.P. and Tsay, B-Y, Using Lotus 1-2-3 macros as grading assistants, J. Accounting. Education(1989), 271--278.
• 9: Dan Gusfield, Algorithms on strings, trees, and sequences: computer science and computational biology, Cambridge University Press, New York, NY, 1997
• 10: Harlen, W (Ed), Enhancing Quality in Assessment, Paul Chapman Publishing, London, 1994.
• 11: Haywood, J. Assessment in Higher Education, Chichester, John Wiley, 1989.
• 12: Joy, M.S. and Luck,M. Plagiarism in Programming Assignments, IEEE Transactions on Education 42(2), (1999), 129--133.
• 13: Long, S. Outcomes of the first live pilot of WP Marker, Internal Report, School of Information Systems, University of East Anglia, June 1999.
• 14: Long, S. Computer-based assessment of authentic word processing skills, Thesis, School of Information Systems, University of East Anglia, Norwich NR4 7TJ, UK, 2001.
• 15: Long.S, and Dowsing, R.D., Building a Computer-based Assessor for IT Skills with Enough Intelligence in Proc Intl Conference on Intelligent Systems and Applications(ISA 2000), (Wollongong, Australia, December 2000), ICSC Academic Press, Switzerland, ISBN 3 906454 24 X.
• 16: Mager R.F. Making Instructions Work, Kogan Page, London, UK, 1990.
• 17: Microsoft Office User Specialist (MOUS), http://www.mous.net/, 2001.
• 18: OPAC, Presenting Solutions Inc., 55 Santa Clara Avenue, Suite 150, Oakland, CA 94610.
• 19: SkillCheck Inc, 113 Terrace Hall Avenue, Burlington, Ma 01803.
• 20: TapDance, PRI Associates, 1905 Chapel Hill Rd, Durham, NC 27707--1171.
• 21: Thomas, A.L., Use of microcomputer spreadsheet software in preparing and grading complex accounting problems, The Accounting Review, 58(4), 777--786.
• 22: Willmott, A.S. and Nuttall, D.L. The reliability of examinations at 16+, Macmillan Education, London, 1975.
• 23: Ziff-Davis, LearnItOnLine, http://www.learnitonline.com/.

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.3 COMPUTERS AND EDUCATION
      K.3.1 Computer Uses in Education

General Terms:
Design, Human Factors

Keywords:
IT skills, automated assessment