Two-Stage Hypotheses Generation for Spoken Language Translation

ABSTRACT

Spoken Language Translation (SLT) is the research area that focuses on the translation of speech or text between two spoken languages. Phrase-based and syntax-based methods represent the state-of-the-art for statistical machine translation (SMT). The phrase-based method specializes in modeling local reorderings and translations of multiword expressions. The syntax-based method is enhanced by using syntactic knowledge, which can better model long word reorderings, discontinuous phrases, and syntactic structure. In this article, we leverage on the strength of these two methods and propose a strategy based on multiple hypotheses generation in a two-stage framework for spoken language translation. The hypotheses are generated in two stages, namely, decoding and regeneration. In the decoding stage, we apply state-of-the-art, phrase-based, and syntax-based methods to generate basic translation hypotheses. Then in the regeneration stage, much more hypotheses that cannot be captured by the decoding algorithms are produced from the basic hypotheses. We study three regeneration methods: redecoding, n-gram expansion, and confusion network in the second stage. Finally, an additional reranking pass is introduced to select the translation outputs by a linear combination of rescoring models. Experimental results on the Chinese-to-English IWSLT-2006 challenge task of translating the transcription of spontaneous speech show that the proposed mechanism achieves significant improvements over the baseline of about 2.80 BLEU-score.

REFERENCES

• 1: Aiello, D., Cerrato, L., Delogu, C., and Di Carlo, A. 1999. EUTRANS project: FUB activity in spoken machine translation. In Proceedings of the Venezia per il Trattamento Automatico delle Lingue (VEXTAL’99).
• 2: Juan Carlos Amengual , Asunción Castaño , Antonio Castellanos , Victor M. Jiménez , David Llorens , Andrés Marzal , Federico Prat , Juan Miguel Vilar , José Miguel Benedi , Francisco Casacuberta , Moisés Pastor , Enrique Vidal, The EuTrans Spoken Language Translation System, Machine Translation, v.15 n.1-2, p.75-103, June 2000 [doi>10.1023/A:1011116115948]
• 3: Bangalore, S., Bordel, G., and Riccardi, G. 2001. Computing consensus translation from multiple machine translation systems. In Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU’01), 351--354.
• 4: Srinivas Bangalore , Giuseppe Riccardi, Stochastic Finite-State Models for Spoken Language MachineTranslation, Machine Translation, v.17 n.3, p.165-184, 2002 [doi>10.1023/B:COAT.0000010804.12581.96]
• 5: Berger, A. L., Brown, P. F., Della Pietra, S. A., Della Pietra, V. J., Kehler, A. S., and Mercer, R. L. 1996. Language translation apparatus and methods using context-based translation models. U.S. Patent No. 5,510,981.
• 6: Besacier, L., Blanchon, H., Fouquet, Y., Guilbaud, J. P., Helme, S., Mazenot, S., Moraru, D., and Vaufreydaz, D. 2001. Speech translation for French in the NESPOLE! European project. In Proceedings of the European Conference on Speech Communication and Technology (EUROSPEECH’01).
• 7: Peter F. Brown , Vincent J. Della Pietra , Stephen A. Della Pietra , Robert L. Mercer, The mathematics of statistical machine translation: parameter estimation, Computational Linguistics, v.19 n.2, June 1993
• 8: Francisco Casacuberta , Enrique Vidal , Juan Miguel Vilar, Architectures for speech-to-speech translation using finite-state models, Proceedings of the ACL-02 workshop on Speech-to-speech translation: algorithms and systems, p.39-44, July 11, 2002 [doi>10.3115/1118656.1118662]
• 9: Chen, B., Cattoni, R., Bertoldi, N., Cettolo, M., and Federico, M. 2005. The ITC-irst SMT System for IWSLT-2005. In Proceedings of the International Workshop for Spoken Language Translation (IWSLT’05), 98--104.
• 10: Chen, B., Cettolo, M., and Federico, M. 2006. Reordering rules for phrase-based statistical machine translation. In Proceedings of the International Workshop for Spoken Language Translation (IWSLT’06).
• 11: Chen, B., Federico, M., and Cettolo, M. 2007a. Better N-best translations through generative n-gram language models. In Proceedings of the Machine Translation Summit XI (MT’07).
• 12: Chen, B., Sun, J., Jiang, H., Zhang M., and Aw, A. T. 2007b. I2R Chinese-English translation system for IWSLT-2007. In Proceedings of the International Workshop for Spoken Language Translation (IWSLT’07).
• 13: Chen, B., Zhang M., Aw, A. T., and Li, H. 2008. Regenerating hypotheses for statistical machine translation. In Proceedings of the International Conference on Computer Linguistics (COLING’08).
• 14: Chen, S. F. and Goodman, J. 1998. An empirical study of smoothing techniques for language modeling. Tech. rep. TR-10-98, Center for Research in Computing Technology, Harvard University.
• 15: George Doddington, Automatic evaluation of machine translation quality using n-gram co-occurrence statistics, Proceedings of the second international conference on Human Language Technology Research, March 24-27, 2002, San Diego, California
• 16: Eurospeech. 2003. Special session: Multilingual speech-to-speech translation. In Proceedings of the 8th European Conference on Speech Communication and Technology (EUROSPEECH’03), 361--384.
• 17: Fellbaum, C. Ed. 1998. WordNet: An Electronic Lexical Database. MIT Press.
• 18: Fiscus, J. G. 1997. A post-processing system to yield reduced word error rates: Recognizer output voting error reduction (ROVER). In Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU’97), 347--352.
• 19: Robert Frederking , Sergei Nirenburg, Three heads are better than one, Proceedings of the fourth conference on Applied natural language processing, October 13-15, 1994, Stuttgart, Germany [doi>10.3115/974358.974380]
• 20: Frederking, R., Rudnicky, A., and Hogan, C. 1997. Interactive speech translation in the DIPLOMAT project. In Proceedings of the Workshop on Spoken Language Translation (IWSLT’97).
• 21: Yuqing Gao , Bowen Zhou , Zijian Diao , Jeffrey Sorensen , Michael Picheny, MARS: A Statistical Semantic Parsing and Generation-Based Multilingual Automatic tRanslation System, Machine Translation, v.17 n.3, p.185-212, 2002 [doi>10.1023/B:COAT.0000010802.38267.29]
• 22: He, Z., Mi, H., Liu, Y., Xiong, D., Luo, W., Huang, Y., Ren, Z., Lu, Y., and Liu, Q. 2007. The ICT statistical machine translation systems for IWSLT 2007. In Proceedings of the International Workshop for Spoken Language Translation (IWSLT’07).
• 23: Hitoshi Iida , Eiichiro Sumita , Osamu Furuse, Spoken-language translation method using examples, Proceedings of the 16th conference on Computational linguistics, August 05-09, 1996, Copenhagen, Denmark [doi>10.3115/993268.993369]
• 24: Hoge, H. 2002. Project proposal TC-STAR: Make speech-to-speech translation real. In Proceedings of the 3rd International Conference on Language Resources and Evaluation (LREC’02), 136--141.
• 25: Horiguchi K. and Franz, A. 1997. A formal basis for spoken language translation by analogy. In Proceedings of the Workshop Spoken Language Translation in Conjunction with ACL/EACL’97 (IWSLT’97), 32--39.
• 26: Eduard Hovy, PANGLOSS: knowledge-based machine translation, Proceedings of the workshop on Human Language Technology, March 08-11, 1994, Plainsboro, NJ [doi>10.3115/1075812.1075948]
• 27: Hsiao, R., Venugopal, A., Kohler, T. Zhang, Y., Zollmann, P. C. A., Vogel, S., Black, A. W., Schultz, T., and Waibel, A. 2006. Optimizing components for handheld two-way speech translation for an English-Iraqi Arabic system. In Proceedings of the International Conference on Spoken Language Processing (ICSLP’06).
• 28: Huang, F. and Papineni, K. 2007. Hierarchical system combination for machine translation. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL’07), 277--286.
• 29: Koehn, P. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of Joint Conference on Empirical Methods in Natural Language Processing (EMNLP’04), 388--395.
• 30: Koehn, P., Axelrod, A., Mayne, A. B., Callison-Burch, C., Osborne, M., and Talbot, D. 2005. Edinburgh system description for the 2005 IWSLT speech translation evaluation. In Proceedings of the Workshop Spoken Language Translation (IWSLT’05).
• 31: Philipp Koehn , Franz Josef Och , Daniel Marcu, Statistical phrase-based translation, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, p.48-54, May 27-June 01, 2003, Edmonton, Canada [doi>10.3115/1073445.1073462]
• 32: Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., and Herbst, E. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the Association of Computer Linguistics (ACL’07), 177--180.
• 33: Olivier Kraif , Boxing Chen, Combining clues for lexical level aligning using the null hypothesis approach, Proceedings of the 20th international conference on Computational Linguistics, p.1261-es, August 23-27, 2004, Geneva, Switzerland [doi>10.3115/1220355.1220538]
• 34: Kumar, S. and Byrne, W. 2004. Minimum Bayes-risk decoding for statistical machine translation. In Proceedings of the Workshop on Human Language Technology (HLT’04).
• 35: Alon Lavie , Chad Langley , Alex Waibel , Fabio Pianesi , Gianni Lazzari , Paolo Coletti , Loredana Taddei , Franco Balducci, Architecture and design considerations in NESPOLE!: a speech translation system for e-commerce applications, Proceedings of the first international conference on Human language technology research, p.1-4, March 18-21, 2001, San Diego [doi>10.3115/1072133.1072140]
• 36: Young-Suk Lee , Wu Sok Yi , Stephanie Seneff , Clifford J. Weinstein, Interlingua-based broad-coverage Korean-to-English translation in CCLINC, Proceedings of the first international conference on Human language technology research, p.1-6, March 18-21, 2001, San Diego [doi>10.3115/1072133.1072193]
• 37: Macherey, W. and Och, F. J. 2007. An empirical study on computing consensus translations from multiple machine translation systems. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL’07), 986--995.
• 38: Matusov, E., Ueffing, N., and Ney, H. 2006. Computing consensus translation from multiple machine translation systems using enhanced hypotheses alignment. In Proceedings of the Conference on the European Chapter of the Association for Computational Linguistic (EACL’06).
• 39: I. Dan Melamed, Models of translational equivalence among words, Computational Linguistics, v.26 n.2, p.221-249, June 2000 [doi>10.1162/089120100561683]
• 40: Ney, H., Nieben, S., Och, F. J., Sawaf, H., Tillmann, C., and Vogel, S. 2000. Algorithms for statistical translation of spoken language. In IEEE Trans. Speech Audio Process. 8, 1, 24--36.
• 41: Ney, H. 2003. The statistical approach to machine translation and a roadmap for speech translation. In Proceedings of European Conference on Speech Communication and Technology (EUROSPEECH’03), 361--364.
• 42: Sergei Nirenburg , Jaime Carbonell , Masaru Tomita , Kenneth Goodman, Machine Translation: A Knowledge-Based Approach, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1994
• 43: Franz Josef Och, Minimum error rate training in statistical machine translation, Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, p.160-167, July 07-12, 2003, Sapporo, Japan [doi>10.3115/1075096.1075117]
• 44: Franz Josef Och , Hermann Ney, Discriminative training and maximum entropy models for statistical machine translation, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073133]
• 45: Franz Josef Och , Hermann Ney, A systematic comparison of various statistical alignment models, Computational Linguistics, v.29 n.1, p.19-51, March 2003 [doi>10.1162/089120103321337421]
• 46: Kishore Papineni , Salim Roukos , Todd Ward , Wei-Jing Zhu, BLEU: a method for automatic evaluation of machine translation, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073135]
• 47: Paul, M., Doi, T., Hwang, Y., Imamura, K., Okuma, H., and Sumita, E. 2005. Nobody is perfect: ATR’s hybrid a roach to spoken language translation. In Proceedings of the Workshop Spoken Language Translation (IWSLT’05), 55--62.
• 48: Chris Quirk , Arul Menezes, Do we need phrases?: challenging the conventional wisdom in statistical machine translation, Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, p.9-16, June 04-09, 2006, New York, New York [doi>10.3115/1220835.1220837]
• 49: Rayner, M. and Bouillon, P. 1995. Hybrid transfer in an English-French spoken language translator. In Proceedings of IA’95, 153--162.
• 50: M. Rayner , D. Carter, Hybrid Language Processing in the Spoken Language Translator, Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97) -Volume 1, p.107, April 21-24, 1997
• 51: Rosti, A., Ayan, N. F., Xiang, B., Matsoukas, S., Schwartz, R., and Dorr, B. 2007a. Combining outputs from multiple machine translation systems. In Proceedings of the Human Language Technology Conference/North American Chapter of the Association for Computational Linguistics (HLT-NAACL’03), 228--235.
• 52: Rosti, A., Matsoukas, S., and Schwartz, R. 2007b. Improved word-level system combination for machine translation. In Proceedings of the Association of Computer Linguistics (ACL’07).
• 53: Sebastian, S., Zong, C., Reichert, J., Cao, W., Kolss, M., Xie, G., Peterson, K., Ding, P., Arranz, V., Yu, J., and Waibel, A. 2006. Speech-to-speech translation services for the Olympic games 2008. In Proceedings of the 3rd Joint Workshop on Machine Learning and Multimodal Interaction (MLMI’06).
• 54: Mark Seligman, Nine Issues in Speech Translation, Machine Translation, v.15 n.1-2, p.149-186, June 2000 [doi>10.1023/A:1011180928513]
• 55: Sim, K. C., Byrne, W. J., Gales, M. J. F., Sahbi, H., and Woodland, P. C. 2007. Consensus network decoding for statistical machine translation system combination. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’07).
• 56: Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of Association for Machine Translation in the Americas (AMTA’06).
• 57: Stolcke, A. 2002. SRILM -- An extensible language modeling toolkit. In Proceedings of the International Conference on Spoken Language (ICSLP’02), 901--904.
• 58: Sugaya, F., Takezawa, T., Yokoo, A., Sagisaka, Y., and Yamamoto, S. 1999. End-to-end evaluation in ATR-MATRIX: Speech translation system between English and Japanese. In Proceedings of European Conference on Speech Communication and Technology (EUROSPEECH’99), 2431--2434.
• 59: Sumita, E., Yamada, S., Yamamoto, K., Paul, M., Kashioka, H., Ishikawa, K., and Shirai S. 1999. Solutions to problems inherent in spoken-language translation: The ATR-MATRIX a roach. In Proceedings of Machine Translation Summit VII (MT’99), 229--235.
• 60: Eiichiro Sumita , Yasuhiro Akiba , Takao Doi , Andrew Finch , Kenji Imamura , Michael Paul , Mitsuo Shimohata , Taro Watanabe, A corpus-centered approach to spoken language translation, Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics, April 12-17, 2003, Budapest, Hungary [doi>10.3115/1067737.1067776]
• 61: Takezawa, T., Sumita, E., Sugaya, F., Yamamoto, H., and Yamamoto, S. 2002. Toward a broad-coverage bilingual corpus for speech translation of travel conversations in the real world. In Proceedings of the International Conference on Language Resources and Evaluation (LREC’02).
• 62: Takezawa, T., Morimoto, T., Sagisaka, Y., Campbell, N., Iida, H., Sugaya, F., Yokoo, A., and Yamamoto, S. 1998. A Japanese-to-English speech translation system: ATR-MATRIX. In Proceedings of the International Conference on Spoken Language Processing (ICSLP’98).
• 63: Christoph Tillmann , Tong Zhang, A localized prediction model for statistical machine translation, Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, p.557-564, June 25-30, 2005, Ann Arbor, Michigan [doi>10.3115/1219840.1219909]
• 64: Nicola Ueffing , Hermann Ney, Word-Level Confidence Estimation for Machine Translation, Computational Linguistics, v.33 n.1, p.9-40, March 2007 [doi>10.1162/coli.2007.33.1.9]
• 65: E. Vidal, Finite-State Speech-to-Speech Translation, Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97) -Volume 1, p.111, April 21-24, 1997
• 66: Vogel, S., Hewavitharana, S., Kolss, M., and Waibel, A. 2004. The ISL statistical translation system for spoken language translation. In Proceedings of the International Workshop on Spoken Language Translation (IWSLT’04).
• 67: A. Waibel , A. N. Jain , A. E. McNair , H. Saito , A. G. Hauptmann , J. Tebelskis, JANUS: a speech-to-speech translation system using connectionist and symbolic processing strategies, Proceedings of the Acoustics, Speech, and Signal Processing, 1991. ICASSP-91., 1991 International Conference, p.793-796, April 14-17, 1991
• 68: Wahlster, W., Ed. 2000. Verbmobil: Foundations of Speech-to-Speech Translations. Springer Verlag, Berlin.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Machine translation

General Terms:
Algorithms, Experimentation, Performance

Keywords:
Spoken language translation, hypotheses generation, statistical machine translation