Object-Oriented Program Comprehension: Effect of Expertise, Task and Phase

ABSTRACT

The goal of our study is to evaluate the effect on program comprehension of three factors that have not previously been studied in a single experiment. These factors are programmer expertise (expert versus novice), programming task (documentation versus reuse), and the development of understanding over time (phase 1 versus phase 2). This study is carried out in the context of the mental model approach to comprehension based on van Dijk and Kintsch's model [(1983) Strategies of Discourse Comprehension. New York: Academic]. One key aspect of this model is the distinction between two kinds of representation the reader might construct from a text: (1) the textbase, which refers to what is said in the text and how it is said, and (2) the situation model, which represents the situation referred to by the text. We have evaluated the effect of the three factors mentioned above on the development of both the textbase (or program model) and the situation model in object-oriented program comprehension. We found a four-way interaction of expertise, phase, task and type of model. For the documentation group we found that experts and novices differ in the elaboration of their situation model but not their program model. There was no interaction of expertise with phase and type of model in the documentation group. For the reuse group, there was a three-way interaction between phase, expertise and type of model. For the novice reuse group, the effect of the phase was to increase the construction of the situation model but not the program model. With respect to the task, our results show that novices do not spontaneously construct a strong situation model but are able to do so if the task demands it.

REFERENCES

• 1: Anderson, J. R. 1976. Language, Memory, and Thought. Hillsdale, NJ: Lawrence Erlbaum Associates.
• 2: Atwood, M. E., and Ramsey, H. R. 1978 Cognitive Structures in the Comprehension and Memory of Computer Programs: An Investigation of Computer Program Debugging. U. S. Army Research Institute for the Behavioral and Social Sciences. Technical report (TR-78-A21), VA: Alexandra.
• 3: David Bergantz , Johnette Hassell, Information relationships in PROLOG programs: how do programmers comprehend functionality?, International Journal of Man-Machine Studies, v.35 n.3, p.313-328, Sept. 1991 [doi>10.1016/S0020-7373(05)80131-2]
• 4: John B. Black , Dana S. Kay , Elliot M. Soloway, Goal and plan knowledge representations: from stories to text editors and programs, Interfacing thought: cognitive aspects of human-computer interaction, MIT Press, Cambridge, MA, 1987
• 5: Deborah A. Boehm-Davis , Robert W. Holt , Alan C. Schultz, The role of program structure in software maintenance, International Journal of Man-Machine Studies, v.36 n.1, p.21-63, Jan. 1992 [doi>10.1016/0020-7373(92)90051-L]
• 6: A. Borgida , S. Greenspan , J. Mylopoulos, Knowledge representations as the basis for requirements specifications, Readings in artificial intelligence and software engineering, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1986
• 7: Bransford, J. D., and Johnson, M. K. 1972. Contextual prerequisites for understanding: some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior 11: 717-726.
• 8: Burkhardt, J.-M. 1997. Réutilisation de Solutions en Conception Orientée-objet: un Modèle Cognitif des Mécanismes et Représentations Mentales (Reuse of Object-Oriented Design Solutions: A Model of Cognitive Process and Representations). Ph.D. Dissertation, Université René Descartes Paris V.
• 9: Burkhardt, J.-M., and Détienne, F. 1995. An empirical study of software reuse by experts in object-oriented design. In: K. Nordby, P. H. Helmersen, D. J. Gilmore, and S. A. Arnesen (eds): Proceedings of INTERACT'95. Chapman & Hall. pp. 133-138.
• 10: Jean-Marie Burkhardt , Françoise Détienne , Susan Wiedenbeck, Mental Representations Constructed by Experts and Novices in Object-Oriented Program Comprehension, Proceedings of the IFIP TC13 Interantional Conference on Human-Computer Interaction, p.339-346, July 14-18, 1997
• 11: Celeux, G., Diday, E., Govaert, G., Lechevallier, Y., and Ralambondrainy, H. 1989. Classification Automatique des Données. Paris: Dunod.
• 12: Cynthia L. Corritore , Susan Wiedenbeck, Mental representations of expert procedural and object-oriented programmers in a software maintenance task, International Journal of Human-Computer Studies, v.50 n.1, p.61-83, Jan. 1999 [doi>10.1006/ijhc.1998.0236]
• 13: Corritore, C. L., and Wiedenbeck, S. 1991. What do novices learn during program comprehension? International Journal of Human-Computer Interaction 3(2): 199-222.
• 14: Détienne, F. 1988. Une Application de la Théorie des Schémas à la Compréhension de Programmes, Le Travail Humain, Numéro Spécial "Psychologie Ergonomique de la Programmation", 51(4): 335-350.
• 15: Détienne, F. 1990. Expert programming knowledge: a schema-based approach. In: J-M. Hoc, T.R.G. Green, R. Samurçay, D. Gilmore (eds): Psychology of Programming. Academic Press, People and Computer Series, pp. 205-222.
• 16: Détienne, F. 1996. What model(s) for program understanding? Proceedings of the Conference on Using Complex Information Systems (UCIS'96), Poitiers, France, September pp. 4-6.
• 17: Détienne, F. 1997. Assessing the cognitive consequences of the object-oriented approach: a survey of empirical research on object-oriented design by individuals and teams. Interacting with Computers 9(1): 47-72.
• 18: Détienne, F., Rouet, J.-F., Burkhardt, J.-M., and Deleuze-Dordron, C. 1996. Reusing processes and documenting processes: toward an integrated framework. In: T. R. G. Green, J. J. Canas, and C. P. Warren (eds): Proceedings of the Eight Conference on Cognitive Ergonomics. ECCE8, Grenada, Spain, September 10-13. pp. 139-144.
• 19: Françoise Détienne, An empirically-derived control structure for the process of program understanding, International Journal of Man-Machine Studies, v.33 n.3, p.323-342, Sept. 1990 [doi>10.1016/S0020-7373(05)80122-1]
• 20: James A. Galambos , John B. Black , Robert P. Abelson, Knowledge Structures, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1986
• 21: Gick, M. L., and Holyoak, K. J. 1983. Schema induction and analogical transfer. Cognitive Psychology 15: 1-38.
• 22: D J. Gilmore , T R.G. Green, Comprehension and recall of miniature programs, International Journal of Man-Machine Studies, v.21 n.1, p.31-48, July 1984 [doi>10.1016/S0020-7373(84)80037-1]
• 23: Green, T. R. G., Petre, M., and Bellamy, R. K. E. 1991. Comprehensibility of visual and textual programs: a test of superlativism against the 'Match-Mismatch' conjecture. In: J. Koenemann-Belliveau, T. G. Moher, and S. P. Robertson (eds.), Empirical Studies of Programmers: Fourth Workshop Norwood, NJ: Ablex. pp. 121-146.
• 24: P. N. Johnson-Laird, Mental models: towards a cognitive science of language, inference, and consciousness, Harvard University Press, Cambridge, MA, 1986
• 25: Kintsch, W. 1974. The Representation of Meaning in Memory. Hillsdale, NJ: Laurence Erlbaum Associates.
• 26: Kintsch, W. 1988. The role of knowledge in discourse comprehension: a comprehension-integration model. Psychological Review: 95(2): 163-182.
• 27: Jürgen Koenemann , Scott P. Robertson, Expert problem solving strategies for program comprehension, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.125-130, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108863]
• 28: B. M. Lange , T. G. Moher, Some strategies of reuse in an object-oriented programming environment, Proceedings of the SIGCHI conference on Human factors in computing systems: Wings for the mind, p.69-73, March 1989
• 29: Adrienne Lee , Nancy Pennington, The effects of paradigm on cognitive activities in design, International Journal of Human-Computer Studies, v.40 n.4, p.577-601, April 1994 [doi>10.1006/ijhc.1994.1028]
• 30: David C. Littman , Jeannine Pinto , Stanley Letovsky , Elliot Soloway, Mental models and software maintenance, Papers presented at the first workshop on empirical studies of programmers on Empirical studies of programmers, p.80-98, June 1986, Washington, D.C., United States
• 31: Mandler, J. M., and Johnson, N. S. 1977. Remembrance of things parsed: story structure and recall. Cognitive Psychology 9: 111-151.
• 32: Mannes, S. 1988. A theoretical interpretation of learning vs. memorizing texts. European Journal of Psychology and Education 3: 157-162.
• 33: Meyer, B. J. F. 1975. The Organization of Prose and its Effect Upon Memory. Amsterdam: North-Holland.
• 34: Mills, B. C., Diehl, V. A., Birkmire, D. P., and Mou, L -C. 1995. Reading procedural texts: effects of purpose for reading and predictions of reading comprehension models. Discourse Processes 20: 79-107.
• 35: Moravcsik, J. and Kintsch, W. 1993. Canadian Journal of Experimental Psychology 47(2): 360-374.
• 36: Norman, D. A., and Rumelhart, D. E. 1975. Explorations in Cognition. San Francisco, CA: W. H. Freeman.
• 37: Nancy Pennington, Comprehension strategies in programming, Empirical studies of programmers: second workshop, Ablex Publishing Corp., Norwood, NJ, 1987
• 38: Pennington, N. 1987b. Stimulus structures and mental representations in expert comprehension of computer programs. Cognitive Psychology 19: 295-341.
• 39: Richard, J. -F. 1990. Connaissances et représentations. In: J. -F. Richard, C. Bonnet, and R. Ghiglione (eds.): Traité de Psychologie Cognitive Vol. 2, Paris: Dunod. pp. 37-102.
• 40: Riecken, R. D., Koenemann-Belliveau, J., and Robertson, S. P. 1991. What do expert programmers communicate by means of descriptive commenting? In: J. Koenemann-Belliveau, T. G. Moher, and S. P. Robertson (eds.): Empirical Studies of Programmers: Fourth Workshop. Norwood, NJ: Ablex. pp. 177- 193.
• 41: Rist, R. S. 1995. Program structure and design. Cognitive Science 19: 507-562.
• 42: Rist, R. 1996. System structure and design. In: W. D. Gray, and D. A. Boehm-Davis (eds): Empirical Studies of Programmers, Sixth workshop. Norwood, NJ: Ablex. pp. 163-194.
• 43: Rosson, M. B., and Alpert, S. R. 1990. The cognitive consequences of object-oriented design. Human-Computer Interaction 5(4): 345-379.
• 44: Mary Beth Rosson , John M. Carroll, Active Programming Strategies in Reuse, Proceedings of the 7th European Conference on Object-Oriented Programming, p.4-20, July 26-30, 1993
• 45: Rouet, J. -F., Deleuze-Dordron, C., and Bisseret, A. 1995. Documentation as part of design: exploratory field studies. In: K. Nordby, P. H. Helmersen, D. J. Gilmore, and S. A. Arnesen (eds.): Proceedings of Interact'95 27-29 June 1995, Lillehammer Norway: Chapman & Hall. pp. 213-216.
• 46: Schank, R. C., and Abelson, R. 1977. Scripts, Plans, Goal, and Understanding. Hillsdale, NJ: Laurence Erlbaum Associates.
• 47: Schmalhofer, F., and Glavanov, D. 1986. Three components of understanding a programmer's manual: verbatim, propositional, and situational representations. Journal of Memory and Language 25: 295-313.
• 48: Schneider, W., Korkel, J., and Weinert, F. E. 1990. Erpert knowledge, general abilities, and text processing. In: W. Schneider and F. E. Weirnert (eds): Interactions Among Aptitudes, Strategies, and Knowledge in Cognitive Performance. New York: Springer Verlag. pp. 235-251.
• 49: Soloway, E., and Ehrlich, K 1984. Empirical studies of programming knowledge. IEEE Transactions on Software Engineering, SE 10,(5): 595-608.
• 50: Elliot Soloway , Kate Ehrlich , Jeffrey Bonar, Tapping into tacit programming knowledge, Proceedings of the 1982 conference on Human factors in computing systems, p.52-57, March 15-17, 1982, Gaithersburg, Maryland, United States [doi>10.1145/800049.801754]
• 51: Bjarne Stroustrup, The C++ programming language (2nd ed.), Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1991
• 52: Tardieu, H., Ehrlich, M.-F., and Gyselinck, V. 1992. levels of representation and domain-specific knowledge in comprehension of scientific texts. Language and Cognitive Processes 7(3/4): 335-351.
• 53: van Dijk, T. A., and Kintsch, W. 1983. Strategies of Discourse Comprehension. New York: Academic.
• 54: Iris Vessey, Toward a theory of computer program bugs: an empirical test, International Journal of Man-Machine Studies, v.30 n.1, p.23-46, January 1989
• 55: Voss, J. F., Fincker-Kiefer, R. H., Greene, T. R., and Post, T. A. 1986. Individual differences in performance: the contrasting approach to knowledge. In: R. J. Sternberg (Ed.): Advances in the Psychology of Human Intelligence Hillsdale, NJ: Erlbaum. pp. 651-667.
• 56: Voss, J. F., Vesonder, G. T., and Spilich, G. J 1980. Text generation and recall by high-knowledge and low-knowledge individuals. Journal of Verbal Learning and Verbal Behavior 10: 651-667.
• 57: S. Wiedenbeck, Beacons in computer program comprehension, International Journal of Man-Machine Studies, v.25 n.6, p.697-709, Dec. 1986 [doi>10.1016/S0020-7373(86)80083-9]
• 58: Susan Wiedenbeck , Vikki Fix , Jean Scholtz, Characteristics of the mental representations of novice and expert programmers: an empirical study, International Journal of Man-Machine Studies, v.39 n.5, p.793-812, Nov. 1993 [doi>10.1006/imms.1993.1084]
• 59: Wiedenbeck, S., Ramalingam, V., Sarasamma, S., and Corritore, C. L. 1999. A comparison of the comprehension of object-oriented and procedural programs by novice programmers. Interacting with Computers 11: 255-282.
• 60: Jeannette M. Wing, A Study of 12 Specifications of the Library Problem, IEEE Software, v.5 n.4, p.66-76, July 1988 [doi>10.1109/52.17803]
• 61: Woodfield, S. N., Embley, D. W., and Scott, D. T. 1987. Can programmers reuse software? IEEE Software July 1987, 52-59.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.9 Management
          Subjects: Productivity

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: Flow charts; Object-oriented design methods

General Terms:
Algorithms, Human Factors, Management

Keywords:
Mental representation, expertise, object-oriented programming, program comprehension, program model, situation model, text comprehension