The rocky road of MPEG-4

The latest MPEG-4 product from Philips is the Streamium MX6000i Internet home entertainment system, which will be introduced early next year.

MPEG-4 is a rather extensive set of definitions for the representation of compressed multimedia content. The open standard defines the concepts of scenes and objects. Objects are components of a scene and are representing a single media element. For example, an object can be a video clip, an audio clip, an image, etc. Objects can be laid out in a predefined manner in terms of time and space in order to create a scene. The standard defines:

• File format for storing multimedia scenes and objects.

• Scene hierarchy description as well as binary representation of objects laid out in a scene in terms of time and space.

• Coded representation of objects.

MPEG-4 was designed in order to enable a number of applications targeted at a variety of rendering devices, such as the TV, set-top box, PC, mobile phone and PDA. As devices vary substantially in capabilities due to different price constraints and different target applications, there was a need to define profiles of MPEG-4 that represent different subsets and capabilities of the overall MPEG-4 standard. As a first step, the following profiles categories were defined:

• Visual profiles: Visual profiles define different profiles that include different subsets of visual (mainly video) compression tools. The simple profiles include only relatively basic compression tools and are, therefore, less efficient. This results in a lower quality-to-bit-rate ratio. They require limited processing resources for playback and encoding, so they are ideal for small, low-price devices such as mobile phones.

• Audio profiles: Audio profiles define different profiles that include different subsets of audio compression tools. The simple profiles in this category are ideal for voice compression, while the more advanced ones are designed for efficient compression of high-quality music.

• Scene graph profiles (included in the systems part of the standard): Scene graph profiles define limitations on the supported complexity of scenes. The simple profiles in this category allow, for example, only one visual object and one audio object in a scene — for example, a video clip with an audio sound track. The more advanced profiles support scenes with a complex hierarchy of objects, including 3D synthetic objects.

Industry acceptance

So far, the industry has adopted in practice only a small portion of the standard as was specified by the Motion Picture Experts Group. MPEG-4 today is mostly referred to as a low-bit-rate low-complexity audio/video format. This is due to the adoption of a subset of its visual and audio profiles.

The first and most important endorsement of MPEG-4 technologies took place in the mobile Internet domain with ETSI-3GPP’s selection of MPEG-4 Simple Visual Profile and Advanced Audio Codec (AAC), the selected music codec for MPEG-4 by ISO-MPEG as optional compression technologies for standard compliant 3G handsets and services. To complement the acceptance of MPEG-4 Simple Visual Profile as an optional compression technology, leading technology vendors were fast enough to react by introducing commercially available mature products based on MPEG-4 technology.

It is important to mention that major concepts and technologies included in the ISO-MPEG specifications of MPEG-4 currently are not making any inroads into the market. MPEG systems profiles (description of scenes and objects into a rich interactive multimedia experience) are not accepted so far by any industry standardization organization or forum. Furthermore, it seems unlikely that MPEG-4 systems will ever reach mass-market deployment.

In order to complement the successful adoption in the mobile industry, Apple, Cisco, IBM, Kasena, Philips and Sun founded the Internet Streaming Media Alliance to define an open-standard streaming media format optimized for broadband wired networks based on MPEG-4.

Intellectual property licensing

MPEG-4 patent owners have created a patent pool and delegated the role of defining and implementing a single licensing program to an organization called MPEG-LA. In 2002, MPEG-LA published the first draft of the license agreement. The license agreement included royalties for a decoder implementation and an encoder implementation targeted at vendors of consumer devices. So far, it is a rather standard licensing program.

However, MPEG-LA introduced usage-based fees targeted at content providers and service operators. The usage-based fees basically require content providers and service operators to pay royalties per subscriber that receives and pays for MPEG-4 content. The licensing terms introduced by MPEG-LA received a lot of heat from the industry. Above all, uncertainties with respect to usage fees (basically, who is required to pay and exactly how much and what kind of services entail royalty payments) cause great concerns in the industry.

In parallel with the work of MPEG on the definition of MPEG-4, the International Telecommunication Union (ITU) started to work on the definition of a video compression standard, called H.263, based on the same patent portfolio as MPEG-4. H.263 was accepted by 3GPP as a mandatory video compression technology for 3G-standard-compliant handsets and services (while MPEG-4 was accepted by 3GPP only as optional).

Even though H.263 is based on an almost identical patent portfolio as MPEG-4, MPEG-LA did not yet publish a patent licensing program for H.263. The outcome is that H.263 is perceived as ‘free’ of royalties and is gaining momentum at the expense of MPEG-4. It is important to note that H.263 is not free, and it is expected that a similar patent licensing program will also be published for this standard.

Conclusions

MPEG-4 is going through a fairly rocky road on its way to become a de-facto industry standard. Educating the market on the patent licensing program and getting into an acceptable compromise between all industry leaders should be a common goal. However, the picture looks promising. MPEG-4 is gaining market share in the mobile domain. This time the ITU and MPEG joined forces in order to define the next generation of MPEG-4 called MPEG-4 AVC or Advanced Video Codec (formerly called H.264).

With support from electronics manufacturers and with an extended technology portfolio (MPEG-4 AVC), MPEG-4 is well-positioned to conquer further territories and become a viable successor to MPEG-2 as the next-generation open standard and de-facto standard technology for streaming media services.

Adi Berenson is director of product managment at Philips Software.