Spoken document retrieval from call-center conversations

ABSTRACT

We are interested in retrieving information from conversational speech corpora, such as call-center data. This data comprises spontaneous speech conversations with low recording quality, which makes automatic speech recognition (ASR) a highly difficult task. For typical call-center data, even state-of-the-art large vocabulary continuous speech recognition systems produce a transcript with word error rate of 30% or higher. In addition to the output transcript, advanced systems provide word confusion networks (WCNs), a compact representation of word lattices associating each word hypothesis with its posterior probability. Our work exploits the information provided by WCNs in order to improve retrieval performance. In this paper, we show that the mean average precision (MAP) is improved using WCNs compared to the raw word transcripts. Finally, we analyze the effect of increasing ASR word error rate on search effectiveness. We show that MAP is still reasonable even under extremely high error rate.

REFERENCES

• 1: M. G. Brown , J. T. Foote , G. J. F. Jones , K. Spärck Jones , S. J. Young, Open-vocabulary speech indexing for voice and video mail retrieval, Proceedings of the fourth ACM international conference on Multimedia, p.307-316, November 18-22, 1996, Boston, Massachusetts, United States [doi>10.1145/244130.244232]
• 2: D. Carmel, E. Amitay, M. Herscovici, Y. S. Maarek, Y. Petruschka, and A. Soffer. Juru at TREC 10 - Experiments with Index Pruning. In Proceedings of the Tenth Text Retrieval Conference (TREC-10). National Institute of Standards and Technology. NIST, 2001.
• 3: David Carmel , Eitan Farchi , Yael Petruschka , Aya Soffer, Automatic query wefinement using lexical affinities with maximal information gain, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564427]
• 4: C. Chelba and A. Acero. Indexing uncertainty for spoken document search. In Interspeech 2005, pages 61--64, Lisbon, Portugal, 2005.
• 5: C. Chelba and A. Acero. Position specific posterior lattices for indexing speech. In Proceedings of the 43rd Annual Conference of the Association for Computational Linguistics (ACL-05), Ann Arbor, MI, 2005. ACL - 2005.
• 6: M. Clements, S. Robertson, and M. Miller. Phonetic searching applied to on-line distance learning modules. In Digital Signal Processing Workshop, 2002 and the 2nd Signal Processing Education Workshop. Proceedings of 2002 IEEE 10th, pages 186--191, 2002.
• 7: J. Garofolo, G. Auzanne, and E. Voorhees. The TREC spoken document retrieval track: A success story. In Proceedings of the Ninth Text Retrieval Conference (TREC-9). National Institute of Standards and Technology. NIST, 2000.
• 8: D. Hakkani-Tur, F. Bechet, G. Riccardi, and G. Tur. Beyond ASR 1-best: Using word confusion networks in spoken language understanding. In Journal of Computer Speech and Language, 2005.
• 9: D. Hakkani-Tur and G. Riccardi. A general algorithm for word graph matrix decomposition. In Proceedings of the IEEE Internation Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 596--599, Hong-Kong, 2003.
• 10: D. James. The application of classical information retrieval techniques to spoken documents. PhD thesis, University of Cambridge, Downing College, 1995.
• 11: L. Mangu, E. Brill, and A. Stolcke. Finding consensus in speech recognition: word error minimization and other applications of confusion networks. Computer Speech and Language, 14(4):373--400, 2000.
• 12: Gilad Mishne , David Carmel , Ron Hoory , Alexey Roytman , Aya Soffer, Automatic analysis of call-center conversations, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099684]
• 13: M. Saraclar and R. Sproat. Lattice-based search for spoken utterance retrieval. In HLT-NAACL 2004: Main Proceedings, pages 129--136, Boston, Massachusetts, USA, 2004.
• 14: A. Singhal, J. Choi, D. Hindle, D. Lewis, and F. Pereira. AT&T at TREC-7. In Proceedings of the Seventh Text Retrieval Conference (TREC-7). National Institute of Standards and Technology. NIST, 1999.
• 15: Amit Singhal , Fernando Pereira, Document expansion for speech retrieval, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.34-41, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312645]
• 16: H. Soltau, B. Kingsbury, L. Mangu, D. Povey, G. Saon, and G. Zweig. The IBM 2004 conversational telephony system for rich transcription. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), March 2005.
• 17: G. Tur, D. Hakkani-Tur, and G. Riccardi. Extending boosting for call classification using word confusion networks. In ICASSP-2004, IEEE International Conference on Acoustics, Speech, and Signal Processing, 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms