Web derived pronunciations for spoken term detection

ABSTRACT

Indexing and retrieval of speech content in various forms such as broadcast news, customer care data and on-line media has gained a lot of interest for a wide range of applications, from customer analytics to on-line media search. For most retrieval applications, the speech content is typically first converted to a lexical or phonetic representation using automatic speech recognition (ASR). The first step in searching through indexes built on these representations is the generation of pronunciations for named entities and foreign language query terms. This paper summarizes the results of the work conducted during the 2008 JHU Summer Workshop by the Multilingual Spoken Term Detection team, on mining the web for pronunciations and analyzing their impact on spoken term detection. We will first present methods to use the vast amount of pronunciation information available on the Web, in the form of IPA and ad-hoc transcriptions. We describe techniques for extracting candidate pronunciations from Web pages and associating them with orthographic words, filtering out poorly extracted pronunciations, normalizing IPA pronunciations to better conform to a common transcription standard, and generating phonemic representations from ad-hoc transcriptions. We then present an analysis of the effectiveness of using these pronunciations to represent Out-Of-Vocabulary (OOV) query terms on the performance of a spoken term detection (STD) system. We will provide comparisons of Web pronunciations against automated techniques for pronunciation generation as well as pronunciations generated by human experts. Our results cover a range of speech indexes based on lattices, confusion networks and one-best transcriptions at both word and word fragments levels.

REFERENCES

• 1: C. Allauzen et al. OpenFST: A general and efficient weighted finite-state library. In CIAA, 2007.
• 2: C. Allauzen, M. Mohri, and M. Saraclar. General-indexation of weighted automata-application to spoken utterance retrieval. In Proc. HLT-NAACL, 2004.
• 3: M. Bisani and H. Ney. Investigations on joint-multigram models for grapheme-to-phoneme conversion. In ICSLP, 2002.
• 4: A. Black, K. Lenzo, and V. Pagel. Issues in building general letter to sound rules. In ESCA WSS-3, 1998.
• 5: U. V. Chaudhari and M. Picheny. Improvements in phone based audio search via constrained match with high order confusion estimates. In Proc. of ASRU, 2007.
• 6: M. Clements, S. Robertson, and M. S. Miller. Phonetic searching applied to on-line distance learning modules. In Proc. of IEEE Digital Signal Processing Workshop, 2002.
• 7: Sabine Deligne , Frédéric Bimbot, Inference of variable-length linguistic and acoustic units by multigrams, Speech Communication, v.23 n.3, p.223-241, Nov. 1997 [doi>10.1016/S0167-6393(97)00048-4]
• 8: Thomas G. Dietterich, Machine Learning for Sequential Data: A Review, Proceedings of the Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition, p.15-30, August 06-09, 2002
• 9: H. Elovitz et al. Letter-to-sound rules for automatic translation of English text to phones. IEEE Trans. ASSP, 1976.
• 10: J. S. Garofolo, C. G. P. Auzanne, and E. M. Voorhees. The trec spoken document retrieval track: A success story. In Proc. of TREC-9, 2000.
• 11: R. Kneser and H. Ney. Improved backing-off for m-gram language modeling. In ICASSP, 1995.
• 12: Jonathan Mamou , Bhuvana Ramabhadran , Olivier Siohan, Vocabulary independent spoken term detection, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277847]
• 13: Yannick Marchand , Robert I. Damper, A multistrategy approach to improving pronunciation by analogy, Computational Linguistics, v.26 n.2, p.195-219, June 2000 [doi>10.1162/089120100561674]
• 14: M. Mohri, F. C. N. Pereira, and M. Riley. Weighted automata in text and speech processing. In Proc. ECAI, Workshop on Extended Finite State Models of Language, 1996.
• 15: NIST. The spoken term detection (std) 2006 evaluation plan, 2006. http://www.nist.gov/speech/tests/std/docs/std06-evalplan-v10.
• 16: S. Parlak and M. Saraclar. Spoken term detection for Turkish broadcast news. In Proc. ICASSP, 2008.
• 17: M. Saraclar and R. Sproat. Lattice-based search for spoken utterance retrieval. In Proc. HLT-NAACL, 2004.
• 18: F. Seide, P. Yu, C. Ma, and E. Chang. Vocabulary-independent search in spontaneous speech. In Proc. of ICASSP, 2004.
• 19: O. Siohan and M. Bacchiani. Fast vocabulary independent audio search using path based graph indexing. In Proc. of Interspeech, 2005.
• 20: H. Soltau, B. Kingsbury, L. Mangu, D. Povey, G. Saon, and G. Zweig. The IBM 2004 conversational telephony system for rich transcription. In Proc. ICASSP, 2005.
• 21: P. Taylor. Hidden Markov models for grapheme to phoneme conversion. In Interspeech, 2005.
• 22: P. C. Woodland , S. E. Johnson , P. Jourlin , K. Spärck Jones, Effects of out of vocabulary words in spoken document retrieval (poster session), Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.372-374, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345661]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

General Terms:
Algorithms, Experimentation, Performance

Keywords:
open vocabulary, pronunciation normalization, spoken term detection, web pronunciation extraction