Extending parameterized problem-tracing questions for Java with personalized guidance

ABSTRACT

Problem-tracing questions are popular among teachers of various programming languages. In an assessment mode these questions allows to evaluate student knowledge of language semantics. In a self-assessment mode, they provide an excellent learning tool. A 2004 ITiCSE working group report [4] stressed the importance of this type of questions to build foundation of higher-level knowledge. Yet the use of problem-tracing questions is still limited due to a large authoring overhead. To resolve this bottleneck, we explored the idea of parameterized question generation [2]. We developed QuizPACK [1], a system which can generate parameterized problem-tracing questions for C programming language. We also developed QuizGuide [1], a personalized guidance system for QuizPACK, which models student knowledge and guides students individually to most appropriate questions to try. The results of our studies demonstrated that QuizPACK strongly benefits student knowledge and that QuizGuide personalized guidance technology increased student ability to answer questions correctly and encouraged them to use the system more extensively (which, in turn, positively impacted their knowledge) [1]. However, parameterized questions in area of C programming were not as diverse from the complexity point of view as parameterized questions explored in other areas such as physics [2]. As a result, it was left unclear whether personalized guidance technology can successfully guide students to a broader range of questions from relatively simple to very difficult.

The work reported in this poster expands our work on parameterized questions to a more sophisticated domain of object-oriented Java programming, which allowed us to introduce questions of much broader. Capitalizing on our experience with QuizPACK, we developed QuizJET (Java Evaluation Toolkit), which supports authoring, delivery, and evaluation of parameterized questions for Java [3]. We also implemented JavaGuide system (Figure 1), which provides personalized guidance for QuizJET questions. We assessed the impact of adaptive navigation support to student work with questions of different complexity as well as the impact of this technology on weaker and stronger students. The results of two classroom studies indicate that personalized guidance encouraged students to use parameterized questions more extensively and also helped them to access right questions at the right time. Students were 2.5 times more likely to answer a quiz correctly with personalized guidance than without such it. In addition, we found that personalized guidance especially benefited weak students to achieve scores comparable with the scores of strong students on each complexity level of questions.

REFERENCES

• 1: Peter Brusilovsky , Sergey Sosnovsky, Engaging students to work with self-assessment questions: a study of two approaches, Proceedings of the 10th annual SIGCSE conference on Innovation and technology in computer science education, June 27-29, 2005, Caparica, Portugal
• 2: E. Kashy , M. Thoennessen , Y. Tsai , N. E. Davis , S. L. Wolfe, Using networked tools to enhance student success rates in large classes, Proceedings of the Frontiers in Education Conference,1997. on 27th Annual Conference. Teaching and Learning in an Era of Change., p.233-237vol.1, November 05-08, 1997
• 3: Hsiao, I., Brusilovsky, P., Sosnovsky, S. Web-based Parameterized Questions for Object-Oriented Programming. in E-Learn 2008. 2008. Las Vegas, USA: AACE.
• 4: Raymond Lister , Elizabeth S. Adams , Sue Fitzgerald , William Fone , John Hamer , Morten Lindholm , Robert McCartney , Jan Erik Moström , Kate Sanders , Otto Seppälä , Beth Simon , Lynda Thomas, A multi-national study of reading and tracing skills in novice programmers, ACM SIGCSE Bulletin, v.36 n.4, December 2004

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.4 Hypertext/Hypermedia
          Subjects: Architectures

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.4 Hypertext/Hypermedia
          Subjects: Navigation

General Terms:
Design, Languages

Keywords:
java, parameterized questions, personalization, self-assessment