A multilingual, multimodal digital video library system

ABSTRACT

This paper presents the iVIEW system, a multi-lingual, multi-modal digital video content management system for intelligent searching and access of English and Chinese video contents. iVIEW allows full content indexing, searching and retrieval of multi-lingual text, audio and video material. It consists image processing techniques for scenes and scene changes analyses, speech processing techniques for audio signal transcriptions, and multi-lingual natural language processing techniques for word relevance determination. iVIEW can host multi-lingual contents and allow multi-modal search. It facilitate content developers to perform multi-modal information processing of rich video media and to construct XML-based multimedia representation in enhancing multi-modal indexing and searching capabilities, so that the end users can enjoy viewing flexible and seamless delivery of multimedia contents in various browsing tools and devices.

REFERENCES

• 1: M. Christel, H.D. Wactlar, S. Stevens, R. Reddy, M. Mauldin, and T. Kanade, "Techniques for the Creation and Exploration of Digital Video Libraries" Multimedia Tools and Applications (Volume 2), Borko Furht, editor. Boston, MA: Kluwer Academic Publishers, 1996
• 2: L.R. Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition", Proc. IEEE, pp. 257--286, February 1989
• 3: P.C. Woodland, T. Hain, S.E. Johnson, T.R. Niesler, A. Tuerk, and S.J. Young, "Experiments in broadcast news transcription", Proc. IEEE, pp. 909--912, vol. 2, May 1998
• 4: H.M. Meng, P.Y. Hui, "Spoken document retrieval for the languages of Hong Kong", Proceedings of 2001 International Symposium on Multimedia Processing, pp. 201--204, May 2001
• 5: D.M. Lovekin, R.E. Yantorno, K.R. Krishnamachari, "Developing usable speech criteria for speaker identification technology", Proc. IEEE, pp. 421--424, vol. 1, May 2001
• 6: S. Mori, C.Y. Suen, and K. Yamamoto, "Historical review of OCR research and development," Proceedings of the IEEE , Volume: 80, Issue: 7, July 1992, Page(s): 1029--1058
• 7: G. Nagy, "At the frontiers of OCR," Proceedings of the IEEE, Volume: 80, Issue: 7 , July 1992, Page(s): 1093--1100
• 8: Jaehwa Park , Venu Govindaraju , Sargur N. Srihari, OCR in a Hierarchical Feature Space, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.4, p.400-407, April 2000 [doi>10.1109/34.845383]
• 9: Yihong Xu , George Nagy, Prototype Extraction and Adaptive OCR, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.21 n.12, p.1280-1296, December 1999 [doi>10.1109/34.817408]
• 10: M.D. Ganis, C.L. Wilson, and J.L. Blue, "Neural network-based systems for handprint OCR applications," IEEE Transactions on Image Processing, Volume: 7 Issue: 8 , Aug. 1998, Page(s): 1097--1112
• 11: Huiping Li, D. Doermann, and O. Kia, "Automatic text detection and tracking in digital video" IEEE Transactions on Image Processing, Volume: 9 Issue: 1 , Jan. 2000, Page(s): 147--156
• 12: Victor Wu , Raghavan Manmatha , Edward M. Riseman, Sr., TextFinder: An Automatic System to Detect and Recognize Text In Images, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.21 n.11, p.1224-1229, November 1999 [doi>10.1109/34.809116]
• 13: Ullas Gargi , David Crandall , Sameer Antani , Tarak Gandhi , Ryan Keener , Rangachar Kasturi, A System for Automatic Text Detection in Video, Proceedings of the Fifth International Conference on Document Analysis and Recognition, p.29, September 20-22, 1999
• 14: C. Garcia, and X. Apostolidis, "Text detection and segmentation in complex color images," Proceedings 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '00, Volume: 4 , 2000, Page(s): 2326--2329
• 15: Lalitha Agnihotri , Nevenka Dimitrova, Text Detection for Video Analysis, Proceedings of the IEEE Workshop on Content-Based Access of Image and Video Libraries, p.109, June 22-22, 1999
• 16: M. A. Turk and A. P. Pentland, "Face Recognition Using Eigenfaces," Proceedings IEEE Computer Society Conf. Computer Vision and Pattern Recognition, Maui, Hawaii, 1991, pp. 586--591
• 17: Tai Sing Lee, Image Representation Using 2D Gabor Wavelets, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.18 n.10, p.959-971, October 1996 [doi>10.1109/34.541406]
• 18: Laurenz Wiskott , Jean-Marc Fellous , Norbert Krüger , Christopher von der Malsburg, Face Recognition by Elastic Bunch Graph Matching, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.775-779, July 1997 [doi>10.1109/34.598235]
• 19: Peter N. Belhumeur , João P. Hespanha , David J. Kriegman, Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.711-720, July 1997 [doi>10.1109/34.598228]
• 20: Jun Zhang, Yong Yan, and Martin Lades, "Face Recognition: Eigenface, Elastic Matching, and Neural Nets," Proceedings of the IEEE, Vol. 85, No. 9, September 1997
• 21: M.N. Wallick, N. da Vitoria Lobo, M. Shah, "A system for placing videotaped and digital lectures on-line", Proceedings of 2001 International Symposium on Multimedia Processing, pp. 461--464, May 2001
• 22: M. Viswanathan, H.S.M. Beigi, A. Tritschler, F. Maali,"Information access using speech, speaker and face recognition", IEEE International Conference on Multimedia and Expo, pp. 493--496, vol. 1, July-August 2000, New York
• 23: Ricky Houghton, Named Faces: Putting Names to Faces, IEEE Intelligent Systems, v.14 n.5, p.45-50, September 1999 [doi>10.1109/5254.796089]
• 24: Howard D. Wactlar , Takeo Kanade , Michael A. Smith , Scott M. Stevens, Intelligent Access to Digital Video: Informedia Project, Computer, v.29 n.5, p.46-52, May 1996 [doi>10.1109/2.493456]
• 25: Michael G. Christel , Alexander G. Hauptmann , Adrienne S. Warmack , Scott A. Crosby, Adjustable Filmstrips and Skims as Abstractions for a Digital Video Library, Proceedings of the IEEE Forum on Research and Technology Advances in Digital Libraries, p.98, March 19-21, 1999
• 26: Michael G. Christel , Andreas M. Olligschlaeger , Chang Huang, Interactive Maps for a Digital Video Library, IEEE MultiMedia, v.7 n.1, p.60-67, January 2000 [doi>10.1109/93.839312]
• 27: Michael G. Christel , Bryan Maher , Andrew Begun, XSLT for tailored access to a digtal video library, Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, p.290-299, January 2001, Roanoke, Virginia, United States [doi>10.1145/379437.379686]
• 28: Wing Hang Cheung , Michael R. Lyu , Kam Wing Ng, Integrating digital libraries by CORBA, XML and Servlet, Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, p.472, January 2001, Roanoke, Virginia, United States [doi>10.1145/379437.379807]
• 29: H.D. Wactlar, "Informedia - Search and Summarization in the Video Medium," Imagina 2000 Conference, Monaco, January 31--February 2, 2000

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.7 Digital Libraries
          Subjects: Standards

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.7 Digital Libraries
          Subjects: User issues; Systems issues

General Terms:
Design, Management

Keywords:
applications, browser on mobile devices, middleware and browser interactions, multi-modal interactions, multimedia management and support