Audio Puzzler: piecing together time-stamped speech transcripts with a puzzle game

ABSTRACT

We have developed an audio-based casual puzzle game which produces a time-stamped transcription of spoken audio as a by-product of play. Our evaluation of the game indicates that it is both fun and challenging. The transcripts generated using the game are more accurate than those produced using a standard automatic transcription system and the time-stamps of words are within several hundred milliseconds of ground truth.

REFERENCES

• 1: Heather Desurvire , Martin Caplan , Jozsef A. Toth, Using heuristics to evaluate the playability of games, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986102]
• 2: Haubold, A. and Kender, J. R., Alignment of Speech to Highly Imperfect Text Transcriptions. in International Conference on Multimedia and Expo (ICME), (2007), 224--227.
• 3: Huang, X., Acero, A. and Hon, H. W. Spoken Language Processing. Prentice Hall, 2001.
• 4: Raph Koster , Will Wright, A Theory of Fun for Game Design, Paraglyph Press, 2004
• 5: Thomas W. Malone, Heuristics for designing enjoyable user interfaces: Lessons from computer games, Proceedings of the 1982 conference on Human factors in computing systems, p.63-68, March 15-17, 1982, Gaithersburg, Maryland, United States [doi>10.1145/800049.801756]
• 6: Miller, G. The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information. The Psychological Review, 63. 81--97.
• 7: Cosmin Munteanu , Ronald Baecker , Gerald Penn , Elaine Toms , David James, The effect of speech recognition accuracy rates on the usefulness and usability of webcast archives, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124848]
• 8: David Pinelle , Nelson Wong , Tadeusz Stach, Heuristic evaluation for games: usability principles for video game design, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357282]
• 9: Ben Shneiderman, Designing for fun: how can we design user interfaces to be more fun?, interactions, v.11 n.5, September + October 2004 [doi>10.1145/1015530.1015552]
• 10: Stark, L., Whittaker, S. and Hirschberg, J., ASR Satisficing: The effects of ASR accuracy on speech retrieval. in Proceedings of International Conference on Spoken Language Processing, (2000).
• 11: Turnbull, D., Liu, R., Barrington, L. and Lanckriet, G., A Game-Based Approach for Collecting Semantic Annotations of Music. in International Symposium on Music Information Retrieval, (2007).
• 12: Luis von Ahn , Laura Dabbish, Labeling images with a computer game, Proceedings of the SIGCHI conference on Human factors in computing systems, p.319-326, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985733]
• 13: Yuan, J., Liberman, M. and Cieri, C., Towards an Integrated Understanding of Speaking Rate in Conversation. in Proceedings of the Conference on Spoken Language Processing, (2006).

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

General Terms:
Design, Human Factors

Keywords:
game, human computation, puzzle, speech transcription