Musical program auralization: Empirical studies

ABSTRACT

Program auralization aims to communicate information about program state, data, and behavior using audio. We have argued that music offers many advantages as a communication medium [Alty 1995]. The CAITLIN system [Alty and Vickers 1997; Vickers 1999; Vickers and Alty 1996, 1998] was constructed to provide auralizations within a formal structured musical framework. Pilot studies [Alty and Vickers 1997; Vickers 1999] showed that programmers could infer program structure from auralizations alone. A study was conducted using 22 novice programmers to assess (i) whether novices could understand the musical auralizations and (ii) whether the musical experience and knowledge of subjects affected their performance. The results show that novices could interpret the auralizations (with accuracy varying across different levels of abstraction) and that musical knowledge had no significant effect on performance. A second experiment was conducted with another 22 novice programmers to study the effects of musical program auralization on debugging tasks. The experiment aimed to determine whether auralizations would lead to higher bug detection rates. The results indicate that, in certain circumstances, musical auralizations can be used to help locate bugs in programs and that musical skill does not affect the ability to make use of the auralizations. In addition, the experiment showed that subjective workload increased when the musical auralizations were used.

REFERENCES

• 1: James L. Alty, Can we use music in computer-human communication?, Proceedings of the HCI'95 conference on People and computers X, p.409-423, April 1996, Huddersfield, United Kingdom
• 2: James L. Alty , Dimitrios I. Rigas, Communicating graphical information to blind users using music: the role of context, Proceedings of the SIGCHI conference on Human factors in computing systems, p.574-581, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274721]
• 3: Alty, J. L. and Vickers, P. 1997. The CAITLIN auralization system: Hierarchical leitmotif design as a clue to program comprehension. In Proceedings of the International Conference on Auditory Display (ICAD'97). Xerox PARC, Palo Alto, CA 94304, Palo Alto. 89--96.
• 4: Alty, J., Bergan, J., and Schepens, A. 1994. The design of the PROMISE multimedia system and its use in a chemical plant. Tech. Rep. 94/P/0112, Loughborough University.
• 5: LISTEN: A Tool to Investigate the Use of Sound for the Analysis of Program Behavior, Proceedings of the 19th International Computer Software and Applications Conference (COMPSAC'95), p.184, August 09-11, 1995
• 6: Bock, D. S. 1994. ADSL: An auditory domain specification language for program auralization. In Proceedings of the International Conference on Auditory Display (ICAD '94), G. Kramer and S. Smith, Eds. Santa Fe Institute, Santa Fe, NM. 251--256.
• 7: Marc H. Brown , John Hershberger, Color and Sound in Algorithm Animation, Computer, v.25 n.12, p.52-63, December 1992 [doi>10.1109/2.179117]
• 8: Hart, S. and Staveland, L. 1988. Development of NASA-TLX (task load index): Results of empirical and theoretical research. In Human Mental Workload, P. Hancock and N. Meshkati, Eds. North Holland, Amsterdam. 139--183.
• 9: Jameson, D. H. 1994. Sonnet: Audio-enhanced monitoring and debugging. In Auditory Display, G. Kramer, Ed. Santa Fe Institute, Studies in the Sciences of Complexity Proceedings. Addison-Wesley, Reading, MA. 253--265.
• 10: Meehan, D. 1993. Knowledge-Based Advising in the Domain of Scientific Programming. Ph.D. thesis, University of Liverpool.
• 11: Meehan, D., Leonard, J., and Schonfelder, J. L. 1991. An intelligent FORTRAN adviser for postgraduates and researchers. In CALISCE 91 International Conference on Computer Aided Learning and Instruction in Science and Engineering. Fred, Lausanne, Switzerland. 409--415.
• 12: Murthi Nanja , Curtis R. Cook, An analysis of the on-line debugging process, Empirical studies of programmers: second workshop, Ablex Publishing Corp., Norwood, NJ, 1987
• 13: NASA Human Performance Research Group 1987. Task load index (NASA TLX) v1.0 computerised version.
• 14: Roger S. Pressman, Software engineering (3rd ed.): a practitioner's approach, McGraw-Hill, Inc., New York, NY, 1992
• 15: Rimmer, A., Pardoe, J., and Vickers, P. 1995. Interactive program assessment using SPROUT. In 3rd Annual Conference on the Teaching of Computing. S. Alexander and P. Magee, Eds. CTI/Dublin City University, Dublin, Ireland. 285--294.
• 16: Vickers, P. 1999. CAITLIN: Implementation of a Musical Program Auralisation System to Study the Effects on Debugging Tasks as Performed by Novice Pascal Programmers. Ph.D. thesis, Loughborough University.
• 17: Vickers, P. and Alty, J. L. 1996. CAITLIN: A musical program auralization tool to assist novice programmers with debugging. In Proceedings of the International Conference on Auditory Display (ICAD '96). S. P. Frysinger and G. Kramer, Eds. Xerox PARC, Palo Alto, CA. 17--24.
• 18: Vickers, P. and Alty, J. L. 1998. Towards some organising principles for musical program auralization. In Proceedings of the International Conference on Auditory Display (ICAD '98). S. A. Brewster and A. D. N. Edwards, Eds. Electronic Workshops in Computing. British Computer Society, Glasgow.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Auditory (non-speech) feedback

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Debugging aids
      D.2.6 Programming Environments

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Audio input/output
      H.5.5 Sound and Music Computing
          Subjects: Methodologies and techniques

General Terms:
Human Factors, Languages

Keywords:
Music, Pascal, auralization, debugging