Phonetic confusion matrix based spoken document retrieval

ABSTRACT

Combined word-based index and phonetic indexes have been used to improve the performance of spoken document retrieval systems primarily by addressing the out-of-vocabulary retrieval problem. However, a known problem with phonetic recognition is its limited accuracy in comparison with word level recognition. We propose a novel method for phonetic retrieval in the CueVideo system based on the probabilistic formulation of term weighting using phone confusion data in a Bayesian framework. We evaluate this method of spoken document retrieval against word-based retrieval for the search levels identified in a realistic video-based distributed learning setting. Using our test data, we achieved an average recall of 0.88 with an average precision of 0.69 for retrieval of out-of-vocabulary words on phonetic transcripts with 35% word error rate. For in-vocabulary words, we achieved a 17% improvement in recall over word-based retrieval with a 17% loss in precision for word error rites ranging from 35 to 65%.

REFERENCES

• 1: Amir, A., Ponceleon, D., Blanchard, B., Petkovic, D., Srinivasan, S. and Cohen, G. Using Audio Time Scale Modification for Video Browsing, in Proceedings of HICSS-33, Hawaii, Jan. 2000.
• 2: Arnon Amir , Savitha Srinivasan , Dulce Ponceleon , Dragutin Petkovic, CueVideo (demonstration abstract): automated video/audio indexing and browsing, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.326, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312759]
• 3: Dharanipragada, S., Franz, M. and Roukos, S. Audio-Indexing For Broadcast News. In Proceedings of Seventh Text Retrieval Conference, TREC-6, (NIST Special Publication) 1997.
• 4: Dharanipragada, S., and Roukos, S. A Fast vocabulary independent algorithm for spotting words in speech. In Proceedings of lCASSP 98, 1998.
• 5: Robert Fung , Brendan Del Favero, Applying Bayesian networks to information retrieval, Communications of the ACM, v.38 n.3, p.42-ff., March 1995 [doi>10.1145/203330.203340]
• 6: Garofolo, J.,Voorhees, E., Auzanne, C., Stanford, V. and Lund, B. (1997). The TREC-7 Spoken Document Retrieval Track Overview and Results. In Proceedings of the seventh Text Retrieval Conference (TREC-7), pp. 79. NIST Special Publication 500-242.
• 7: James, D. System for Unrestricted Topic Retrieval from Radio News Broadcasts, In Proceedings of ICASSP-96, Atlanta, GA, May196, pp. 279-282.
• 8: Bernard J. Jansen , Amanda Spink , Judy Bateman , Tefko Saracevic, Real life information retrieval: a study of user queries on the Web, ACM SIGIR Forum, v.32 n.1, p.5-17, Spring 1998 [doi>10.1145/281250.281253]
• 9: Johnson, S.E., Jourlin, P., Moore, G.L., Jones, K.S. and Woodland, P.C. Spoken Document Retrieval for TREC-7 at Cambridge University. In Proceedings of the Seventh Text Retrieval Conference (TPREC-7), (NIST Special Publication) 1998
• 10: Jones, G. J. F., Foote, J. T., Jones, K. S., and Young, S. J.. Video Mail Retrieval: the effect of word spotting accuracy on precision. In Proceedings of ICASSP 95, volume 1, pp. 309-312, Detroit, MI.
• 11: G. J. F. Jones , J. T. Foote , K. Spärck Jones , S. J. Young, Retrieving spoken documents by combining multiple index sources, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.30-38, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243208]
• 12: Jones, K. S., Walker, S. and Robertson, S.E. A probabilistic model of information retrieval: Develepment and STatus, TR 446, Cambridge University Computer Laboratory, Sept 1998.
• 13: See URL at http://www.lotus.com/home.nsf/tabs/learnspace
• 14: Lunassen, L.M. and Mercer, R.L. An Information Theoretic Approach to Automatic Determination of Phonemic Baseforms. In Proceedings of ICASSP 84, pp. 42.5.1-42.5.4, 1984.
• 15: M. E. Maron , J. L. Kuhns, On Relevance, Probabilistic Indexing and Information Retrieval, Journal of the ACM (JACM), v.7 n.3, p.216-244, July 1960 [doi>10.1145/321033.321035]
• 16: Ng, K. and Zue, V. Phonetic Recognition for Spoken Document Retrieval. In Proceedings of ICASSP 98, pp. 325-328.
• 17: Robertson, S.E. and SparckoJones, K. Relevance weighting of search terms. In Journal of American Society of Information Sciences. 27 (May-June 1976). pp. 126-146.
• 18: Robertson, S.E., Walker, A., Sparck-Jones, K., Hancock-Beaulieu M.M & Gatford, M. Okapi at TREC-3. In Prec. Third Text Retrieval Conference. (NIST special publication), 1995.
• 19: Sch/tuble, P. and Wechsler, M. First Experiences with a System for Content Based Retrieval of Information from Speech Recordings. In IJCAI-95, Workshop on Intelligent Multimedia Information Retrieval, Maybury, M.T.
• 20: Siegler, M.A., Witbrock, M.J., Slattery, S.T., Seymore, K., Jones, R.E. and Hauptmann, A.G. Experiments in Spoken Document Retrieval at CMU. In Ptvceedings of the Seventh Text Retrieval Conference (TREC-7), (NIST Special Publication) 1998.
• 21: Singhal, A., Col, J., Hindle, D., Lewis, D. and Pereira, F. AT&T at TREC-7. In Proceedings of the Seventh Text Retrieval Conference TREC-7, (NIST Special Publication) 1998.
• 22: Savitha Srinivasan , Dulce Ponceleon , Dragutin Petkovic , Mahesh Viswanathan, Query Expansion for Imperfect Speech: Applications in Distributed Learning, Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'00), p.50, June 16-16, 2000
• 23: See URL at http://cwp.stanford.edu.
• 24: See URL at http://www-4.ibm.com/software/speecld
• 25: Voorhees, E., Garofolo, J. and Jones, K. (1997). The TREC-6 Spoken Document Retrieval Track Overview and Results. In Proceedings of the sixth Text Retrieval Conference (TREC-6), pp. 83. NIST Special Publication 500-240.
• 26: Martin Wechsler , Eugen Munteanu , Peter Schäuble, New techniques for open-vocabulary spoken document retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.20-27, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290950]
• 27: Michael J. Witbrock , Alexander G. Hauptmann, Using words and phonetic strings for efficient information retrieval from imperfectly transcribed spoken documents, Proceedings of the second ACM international conference on Digital libraries, p.30-35, July 23-26, 1997, Philadelphia, Pennsylvania, United States [doi>10.1145/263690.263779]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Speech recognition and synthesis

Additional Classification:
  G. Mathematics of Computing

General Terms:
Design, Experimentation, Performance, Reliability, Theory