What novice programmers don't know

ABSTRACT

Novice programmer knowledge contains a mixture of well-formed, in-transition and muddled conceptual structures. In this paper we describe an analysis of the in-transition and muddled items that are not fully integrated into the novices' cognitive structures. When participants were asked to perform card sorts of programming concepts into categories, 23% of their categories were "ragbags": categories with names such as "don't know," "not sure," or "not applicable"'' that indicate that the students have little or no knowledge of the concepts placed in those categories.In this study, we find that there are distinct differences in the uses of the ragbags. In particular, we find that terms considered more abstract tend to be placed into Don't Know and Not Sure ragbags more often than concrete terms; and students categorized as low performers tend to use Not Sure far more often than high performers but Don't Know and Not Applicable less often. We also find evidence that the meaningfulness of a concept is likely to be related to the vocabulary used in the classroom, suggesting that students may assimilate abstract concepts into their conceptual structures more quickly if one uses the terms more frequently.

REFERENCES

• 1: B. Adelson. Problem solving and the development of abstract categories in programming languages. Memory and Cognition, 9(4):422--433, 1981.
• 2: C. M. Allwood, Novices on the computer: a review of the literature, International Journal of Man-Machine Studies, v.25 n.6, p.633-658, Dec. 1986 [doi>10.1016/S0020-7373(86)80079-7]
• 3: A. M. Burton , N. R. Shadbolt , A. P. Hedgecock , G. Rugg, A formal evaluation of knowledge elicitation techniques for expert systems: domain 1, Proceedings of Expert Systems '87 on Research and Development in Expert Systems IV, p.136-145, January 1988, Brighton, United Kingdom
• 4: M. T. H. Chi, R. Glazer, and M. J. Farr, editors. The nature of expertise. Erlbaum, 1988.
• 5: W. J. Conover. Practical Nonparametric Statistics. Wiley series in probability and statistics. John Wiley & Sons, Inc., New York, NY, 1971.
• 6: S. P. Davies, D. J. Gilmore, and T. R. G. Green. Are objects that important? the effects of expertise and familiarity on the classification of object-oriented code. Human-Computer Interaction, 10(2 & 3):227--248, 1995.
• 7: M. W. Eysenck and M. T. Keane. A Handbook of Cognitive Psychology. Psychology Press, 1995.
• 8: G. Kelly. The psychology of personal constructs. Norton, 1955.
• 9: Tracy L. Lewis , Mary Beth Rosson , Manuel A. Pérez-Quiñones, What do the experts say?: teaching introductory design from an expert's perspective, Proceedings of the 35th SIGCSE technical symposium on Computer science education, March 03-07, 2004, Norfolk, Virginia, USA
• 10: N. A. M. Maiden , M. Hare, Problem domain categories in requirements engineering, International Journal of Human-Computer Studies, v.49 n.3, p.281-304, Sept. 1998 [doi>10.1006/ijhc.1998.0206]
• 11: F. Marton and R. Säljö. On qualitative differences in learning: Outcome and process. British Journal of Educational Psychology, 46:4--11, 1976.
• 12: Michael McCracken , Vicki Almstrum , Danny Diaz , Mark Guzdial , Dianne Hagan , Yifat Ben-David Kolikant , Cary Laxer , Lynda Thomas , Ian Utting , Tadeusz Wilusz, A multi-national, multi-institutional study of assessment of programming skills of first-year CS students, ACM SIGCSE Bulletin, v.33 n.4, December 2001 [doi>10.1145/572139.572181]
• 13: P. McGeorge and G. Rugg. The uses of "contrived" knowledge elicitation techniques. Expert Systems, 9:149--154, 1992.
• 14: Laurie Murphy , Renée McCauley , Suzanne Westbrook , Timothy Fossum , Susan Haller , Briana B. Morrison , Brad Richards , Kate Sanders , Carol Zander , Ruth E. Anderson, A multi-institutional investigation of computer science seniors' knowledge of programming concepts, Proceedings of the 36th SIGCSE technical symposium on Computer science education, February 23-27, 2005, St. Louis, Missouri, USA
• 15: M. Petre, S. Fincher, J. Tenenberg, R. Anderson, R. Anderson, D. Bouvier, S. Fitzgerald, A. Gutschow, S. Haller, G. Lewandowski, R. Lister, R. McCauley, J. McTaggart, B. Morrison, L. Murphy, C. Prasad, B. Richards, K. Sanders, T. Scott, D. Shinners-Kennedy, L. Thomas, S. Westbrook, and C. Zander. 'My criterion is: Is it a boolean.' A cardsort elicitation of students' knowledge of programming constructs. Technical Report 6-03, Computing Laboratory, University of Kent, Canterbury, UK, 2003.
• 16: Christine Prasad , Xiaosong Li, Teaching introductory programming to Information Systems and Computing majors: is there a difference?, Proceedings of the sixth conference on Australasian computing education, p.261-267, January 01, 2004, Dunedin, New Zealand
• 17: G. Rugg and P. McGeorge. The sorting techniques: Card sorts, picture sorts and item sorts. Expert Systems, 14(2):80--93, 1997.
• 18: G. Rugg, P. McGeorge, and N. Maiden. Method fragments. Expert Systems, 17:248--257, 2000.
• 19: K. Sanders, D. Bouvier, S. Fincher, G. Lewandowski, B. Morrison, L. Murphy, M. Petre, J. Tenenberg, L. Thomas, R. Anderson, R. Anderson, S. Fitzgerald, A. Gutschow, S. Haller, M. Jadud, R. Lister, R. McCauley, J. McTaggart, C. Prasad, B. Richards, T. Scott, D. Shinners-Kennedy, S. Westbrook, and C. Zander. A multi-institutional, multinational study of programming concepts using card sort data. Expert Systems, 22(3):121--128, 2005.
• 20: Linda Upchurch , Gordon Rugg , Barbara Kitchenham, Using Card Sorts to Elicit Web Page Quality Attributes, IEEE Software, v.18 n.4, p.84-89, July 2001 [doi>10.1109/MS.2001.936222]

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.3 COMPUTERS AND EDUCATION
      K.3.2 Computer and Information Science Education

General Terms:
Experimentation, Measurement, Performance

Keywords:
CS1, CS2, abstraction, card sorts, course performance, programming concepts