An efficient, streamable text format for multimedia captions and subtitles

ABSTRACT

In spite of the high profile of media types such as video, audio and images, many multimedia presentations rely extensively on text content. Text can be used for incidental labels, or as subtitles or captions that accompany other media objects. In a multimedia document, text content is not only constrained by the need to support presentation styles and layout, it is also constrained by the temporal context of the presentation. This involves intra-text and extra text timing synchronization with other media objects. This paper describes a new timed-text representation language that is intended to be embedded in a non-text host language. Our format, which we call aText (for the Ambulant Text Format), balances the need for text styling with the requirement for an efficient representation that can be easily parsed and scheduled at runtime. aText, which can also be streamed, is defined as an embeddable text format for use within declarative XML languages. The paper presents a discussion of the requirements for the format, a description of the format and a comparison with other existing and emerging text formats. We also provide examples for aText when embedded within the SMIL and MLIF languages and discuss our implementation experiences of aText with the Ambulant Player.

REFERENCES

• 1: Adams, G., Timed Text (TT) Authoring Format 1.0 -- Distribution Format Exchange Profile (DFXP), W3C, 2006.
• 2: Berglund, A., Extensible Stylesheet Language (XSL) Version 1.1, W3C, 2006.
• 3: Dick C.A. Bulterman , Jack Jansen , Kleanthis Kleanthous , Kees Blom , Daniel Benden, Ambulant: a fast, multi-platform open source SMIL player, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027646]
• 4: Bulterman, D.C.A et al., Synchronized Multimedia Integration Language (SMIL), W3C, 2005.
• 5: Cruz-Lara, ISO/TC 37/SC 4 N 328 rev00: MLIF, ISO, 2006.
• 6: Davis, M., The Bidirectional Algorithm, Unicode Standard Annex #9, Unicode, Inc. 2006. http://unicode.org/reports/tr9/
• 7: ISO/IEC 14496-17:2006, MPEG-4 Part 17 Timed Text Format, 2006.
• 8: Bulterman, D.C.A. and Jansen, A.J., Defining and Manipulating Document State in Time-Based XML Multimedia presentations. CWI Technical Report. http://homepages.cwi.n/~dcab/PDF/State07.pdf.
• 9: Lie, Cascading Style Sheets, Pearson Addison Wesley, 2004
• 10: Microsoft, Introduction to HTML+TIME, 2002. http://msdn.microsoft.com/workshop/author/behaviors/ time.asp
• 11: Newman, D. et al., XHTML+SMIL Profile, W3C, 2002.
• 12: Pemberton, S. et al., XHTML™ 1.0 The Extensible HyperText Markup Language (Second Edition), W3C, 2002.
• 13: Raggett, D., Hypertext Markup Language (HTML), W3C, 1999.
• 14: RealNetworks, The Real Networks Production Guide, 2004, http://service.real.com/help/library/guides/ProductionGuide/ prodguide/realpgd.htm
• 15: Wikipedia, SubRip: Subtitle Ripping. http://en.wikipedia.org/wiki/SubRip
• 16: 3GPP, Timed text format (Release 6), TS-26245, 2004.

INDEX TERMS

Primary Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications
          Subjects: Specialized application languages

Additional Classification:
  I. Computing Methodologies
  I.7 DOCUMENT AND TEXT PROCESSING
      I.7.2 Document Preparation
          Subjects: Languages and systems

General Terms:
Design, Experimentation, Languages, Performance, Standardization

Keywords:
DFXP, SMIL, ambulant, realtext, streaming text, timed text