Scalable summaries of spoken conversations

ABSTRACT

In this work, we present a novel means of browsing recorded audio conversations. The method we develop produces scalable summaries of the recognized speech, in which we can increase the amount of text continuously with the desired level of detail to best fill the available space. We present an interface in which a user can view an entire conversation in one screen, but can also quickly zoom in to see the full transcript; the corresponding audio can be easily played as well. The scaling is achieved via a combination of topic segmentation and informative phrase selection, where the threshold for informativeness decreases with increasing level of detail. Finally, we evaluate our method and interface against a baseline interface with a user study.

REFERENCES

• 1: B. Bederson, B., Hollan, J. D., Perlin, K., Meyer, J., Bacon, D., & Furnas, G. W. (1996). "Pad++: A Zoomable Graphical Sketchpad for Exploring Alternate Interface Physics." Journal of Visual Languages and Computing, 7, 3--31
• 2: Doug Beeferman , Adam Berger , John Lafferty, Statistical Models for Text Segmentation, Machine Learning, v.34 n.1-3, p.177-210, Feb. 1999 [doi>10.1023/A:1007506220214]
• 3: Alexandra Canavan, David Graff, and George Zipperlen, CALLHOME American English Speech, LDC Catalog Number LDC97S42, Linguistic Data Consortium, Philadelphia, 1997.
• 4: H. Christensen, B. Kolluru, Y. Gotoh and S. Renals, "From Text Summarization to Style-Specific Summarization for Broadcast News." In Proc. of (ECIR'04), Sunderland, UK, 2004.
• 5: Liwei He , Elizabeth Sanocki , Anoop Gupta , Jonathan Grudin, Auto-summarization of audio-video presentations, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.489-498, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319691]
• 6: Marti A. Hearst, TextTiling: segmenting text into multi-paragraph subtopic passages, Computational Linguistics, v.23 n.1, p.33-64, March 1997
• 7: J. Hirschberg, "Speech Summarization." Lecture Slides available at http://www1.cs.columbia.edu/~julia/cs4706/sum.ppt
• 8: C. Hori and S. Furui, "A New Approach to Automatic Speech Summarization." IEEE Transactions on Multimedia, Vol. 5, NO. 3, September 2003, pp. 368--378.
• 9: Kasper Hornbæk , Benjamin B. Bederson , Catherine Plaisant, Navigation patterns and usability of zoomable user interfaces with and without an overview, ACM Transactions on Computer-Human Interaction (TOCHI), v.9 n.4, p.362-389, December 2002 [doi>10.1145/586081.586086]
• 10: W. Hsu, L. Kennedy, S.-F. Chang, M. Franz, J. Smith, "Columbia-IBM News Video Story Segmentation In TRECVID 2004." Columbia ADVENT Technical Report 209-2005-3, 2005.
• 11: Konstantinos Koumpis , Steve Renals, Automatic summarization of voicemail messages using lexical and prosodic features, ACM Transactions on Speech and Language Processing (TSLP), v.2 n.1, p.1-es, February 2005 [doi>10.1145/1075389.1075390]
• 12: L. Lamel and J. L. Gauvain, "Alternate Phone Models for Conversational Speech," Proc. IEEE ICASSP'05, Philadelphia, March 2005.
• 13: H. R. Lindman, Analysis of Variance in Complex Experimental Designs, San Francisco: W. H. Freeman and Co., 1974.
• 14: S. R. Maskey and J. Hirschberg, "Summarizing Speech Without Text Using Hidden Markov Models," in Proceedings of HLT-NAACL, 2006.
• 15: Mark T. Maybury, Discourse cues for broadcast news segmentation, Proceedings of the 17th international conference on Computational linguistics, p.819-822, August 10-14, 1998, Montreal, Quebec, Canada
• 16: Klaus Ries, Segmenting Conversations by Topic, Initiative, and Style, Information Retrieval Techniques for Speech Applications [this book is based on the workshop "Information Retrieval Techniques for Speech Applications", held as part of the 24th Annual International ACM SIGIR Conference on Research and Development in Infor, p.51-66, September 01, 2001
• 17: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 18: Klaus Zechner , Alex Waibel, DiaSumm: flexible summarization of spontaneous dialogues in unrestricted domains, Proceedings of the 18th conference on Computational linguistics, July 31-August 04, 2000, Saarbrücken, Germany [doi>10.3115/992730.992786]
• 19: K. Zechner, "Summarization of Spoken Language - Challenges, Methods, and Prospects," Speech Technology Expert eZine, Issue 6, January 2002.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.3 Communications Applications
          Subjects: Information browsers

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Audio input/output
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Text analysis

Keywords:
conversation browsing, conversation summarization, speech browsing, speech summarization, zoomable user interfaces