MPEG-4: an object-based multimedia coding standard supporting mobile applications

ABSTRACT

The ISO MPEG committee, after successful completion of the MPEG-1 and the MPEG-2 standards is currently working on MPEG-4, the third MPEG standard. Originally, MPEG-4 was conceived to be a standard for coding of limited complexity audio-visual scenes at very low bit-rates; however, in July 1994, its scope was expanded to include coding of scenes as a collection of individual audio-visual objects and enabling a range of advanced functionalities not supported by other standards. One of the key functionalities supported by MPEG-4 is robustness in error prone environments. In general, the MPEG-4 standard provides solutions for coding of natural or synthetic video and audio, as well as a system for multiplex/demultiplex and description of scenes in a truly flexible manner. With focus on the mobile multimedia functionality, we present an overview of the current status as well as the details of the MPEG-4 coding standard. We also discuss profiles, a mechanism used for partitioning MPEG-4 into realizable subsets. Finally, plans for testing and verification of current MPEG-4 (Version 1) standard, ongoing work for MPEG-4 Version 2, as well as directions for MPEG-7, the next MPEG standard, are briefly discussed.

REFERENCES

• 1: Ad Hoc Group on Error Resilience Core Experiments, Plan for March 1998 error resilience verification test, ISO/IEC JTC1/SC29/ WG11 N1829, Stockholm (July 1997).
• 2: Andrea L. Ames , David R. Nadeau , John L. Moreland, The VRML 2.0 sourcebook (2nd ed.), John Wiley & Sons, Inc., New York, NY, 1997
• 3: AOE Group, MPEG-4 proposal package description (PPD) - Rev. 3, ISO/IEC JTC1/SC29/WG11 N0998, Tokyo (July 1995).
• 4: AOE Group, MPEG-4 testing and evaluation procedures document, ISO/IEC JTC1/SC29/WG11 N0999, Tokyo (July 1995).
• 5: O. Avaro, P. Chou, A. Eleftheriadis, C. Herpel and C. Reader, The MPEG-4 system and description languages, Signal Processing: Image Communication (Special issue on MPEG-4), to appear in 1997.
• 6: L. Chiariglione, MPEG-4 call for proposals, ISO/IEC JTC1/ SC29/WG11 N0997, Tokyo (July 1995).
• 7: L. Chiariglione, Resolutions of 40th WG11 meeting, ISO/IEC JTC1/SC29/WG11 N1716, Stockholm (July 1997).
• 8: R. Cox, B. Haskell, Y. LeCun, B. Shahraray and L. Rabiner, On the applications of multimedia processing to communications, AT&T internal technical memo; to appear in IEEE Transactions.
• 9: T. Ebrahimi, Report of ad hoc group on definition of VMs for content based video representation, ISO/IEC JTC1/SC29/WG11 MPEG 96/0642, Munich (January 1996).
• 10: A. Eleftheriadis, The MPEG-4 system description language: From practice to theory, in: Proceedings of 1997 IEEE International Conference on Circuits and Systems ISCAS '97, Hong Kong (June 1997).
• 11: Alexandros Eleftheriadis, Flavor: a language for media representation, Proceedings of the fifth ACM international conference on Multimedia, p.1-9, November 09-13, 1997, Seattle, Washington, United States [doi>10.1145/266180.266319]
• 12: Y. Fang and A. Eleftheriadis, A syntactic framework for bitstreamlevel representation of audio-visual objects, in: Proceedings of 3rd IEEE International Conference on Image Processing ICIP '96,Lausanne, Switzerland (September 1996).
• 13: Flavor Web Site: http://www.ee.columbia.edu/flavor.
• 14: James Gosling , Bill Joy , Guy L. Steele, The Java Language Specification, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1996
• 15: Barry G. Haskell , Atul Puri , Arun N. Netravali, Digital Video: An introduction to MPEG-2, Chapman & Hall, Ltd., London, UK, 1996
• 16: B.G. Haskell, A. Puri and J. Osterman, Happenings in ISO MPEG: An introduction to MPEG-4, invited presentation at Data Compression Conference, Snow Bird (March 1997).
• 17: IEEE Transactions on Circuits and Systems for Video Technology (Special issue on MPEG-4) 7(1) (February 1997).
• 18: ISO/IEC 11172 International Standard (MPEG-1), Information technology - Coding of moving pictures and associated audio for digital storage media at up to about 1,5 Mbit=s (1993).
• 19: ISO/IEC 13818 International Standard (MPEG-2), Information technology - Generic coding of moving pictures and associated audio (also ITU-T Rec. H.262) (1995).
• 20: ISO/IEC 14472 draft international standard: Virtual reality modeling language (1997).
• 21: ISO/IEC JTC1/SC29/WG11 (MPEG) Web Site: http://drogo.cselt.it/ mpeg.
• 22: ITU-T Recommendation H.261, Video codec for audio-visual services at p - 64 kbit=s (1990).
• 23: ITU-T, Draft ITU-T Recommendation H.263: Video coding for low bit-rate communication (December 1995).
• 24: ITU-T Recommendation H.223: Multiplexing protocol for low bitrate multimedia communication (1995).
• 25: ITU-T Recommendation H.223 Annex A: Multiplexing protocol for low bit-rate mobile multimedia communication (1996).
• 26: ITU-T H.263+ Video Group, Draft 12 of ITU-T Recommendation H.263+ (May 1997).
• 27: D. Lindbergh, The H.324 multimedia communication standard, IEEE Communications Magazine 34(12) (December 1996) 46-51.
• 28: MPEG-4 Audio Group, MPEG-4 audio working draft version 4.0, ISO/IEC JTC1/SC29/WG11 N1745, Stockholm (July 1997).
• 29: MPEG-4 Integration Group, MPEG-4 SNHC call for proposals, ISO/IEC JTC1/SC29/WG11 N1195, Florence (March 1996).
• 30: MPEG-4 Integration Group, MPEG-4 SNHC proposal package description, ISO/IEC JTC1/SC29/WG11 N1199, Florence (March 1996).
• 31: MPEG-4 Requirements Ad Hoc Group, Draft of MPEG-4 requirements, ISO/IEC JTC1/SC29/WG11 N1238, Florence (March 1996).
• 32: MPEG-4 SNHC Web Site: http://www.es.com/mpeg4-snhc.
• 33: MPEG-4 Systems Group, MPEG-4 systems working draft version 5.0, ISO/IEC JTC1/SC29/WG11 N1825, Stockholm (July 1997).
• 34: MPEG-4 Systems Web Site: http://garuda.imag.fr/MPEG4.
• 35: MPEG-4 Video and SNHC Groups, MPEG-4 visual working draft version 4.0, ISO/IEC JTC1/SC29/WG11 N1797, Stockholm (July 1997).
• 36: MPEG-4 Video Group, MPEG-4 video verification model version 2.0, ISO/IEC JTC1/SC29/WG11 N1260, Florence (March 1996).
• 37: MPEG-4 Video Group, MPEG-4 video verification model version 7.0, ISO/IEC JTC1/SC29/WG11 N1642, Bristol (April 1997).
• 38: MPEG-4 Video Web Site: http://wwwam.hhi.de/mpeg-video.
• 39: H. Peterson, Report of the ad hoc group on MPEG-4 video testing logistics, ISO/IEC JTC1/SC29/WG11 Doc. MPEG95/0532, (Novem-ber 1995).
• 40: A. Puri, Status and direction of the MPEG-4 standard, in: International Symposium on Multimedia and Video Coding,NewYork (October 1995); also published in a book by Plenum Press.
• 41: A. Puri, Report of ad hoc group on coordination of future core experiments in MPEG-4 video, ISO/IEC JTC1/SC29/WG11 MPEG 96/0669, Munich (January 1996).
• 42: A. Puri, MPEG-4: A flexible and extensible multimedia coding standard in progress, invited paper in IEEE Multimedia book (1996).
• 43: A. Puri, A.R. Reibman, R.L. Schmidt and B.G. Haskell, Robustness considerations in ISO MPEG-4 and ITU-T mobile video standards, in: Proceedings MoMuC-3, Princeton (September 1996) (Plenum Press, 1997).
• 44: A. Puri, R.L. Schmidt and B.G. Haskell, Improvements in DCT- based video coding, in: Proc. SPIE Visual Communications and Image Processing, San Jose (February 1997).
• 45: Requirements Group, MPEG-4 applications document, ISO/IEC JTC1/SC29/WG11 N1729, Stockholm (July 1997).
• 46: Requirements Group, MPEG-4 overview, ISO/IEC JTC1/SC29/ WG11 N1730 (July 1997).
• 47: Requirements Group, MPEG-4 profile requirements version 4, ISO/IEC JTC1/SC29/WG11 N1728, Stockholm (July 1997).
• 48: Requirements Group, MPEG-4 requirements version 4, ISO/IEC JTC1/SC29/WG11 N1727, Stockholm (July 1997).
• 49: Requirements Group, MPEG-7 context and objectives version 4, ISO/IEC JTC1/SC29/WG11 N1733, Stockholm (July 1997).
• 50: Signal Processing: Image Communication (Special issue on MPEG- 4, Part 1: Invited papers) 10(1-3) (May 1997).
• 51: Signal Processing: Image Communication (Special issue on MPEG- 4, Part 2: Submitted papers) 10(4) (July 1997).
• 52: P. J. L. Van Beek , A. M. Tekalp , A. Puri, 2-D Mesh Geometry and Motion Compression for Efficient Object-Based Video Representation, Proceedings of the 1997 International Conference on Image Processing (ICIP '97) 3-Volume Set-Volume 3, p.440, October 26-29, 1997

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.0 General
          Subjects: Data communications

  E. Data

  K. Computing Milieux
  K.1 THE COMPUTER INDUSTRY
      Subjects: Standards

General Terms:
Algorithms, Design, Standardization