Standards conversion
Dec 1, 2007 12:00 PM, By David Tasker
Get content in the right frame rate every time.
The original purpose of the standards converter was to address the incompatibilities between the world's multiple television standards. Dealing with standards featuring different frame and field rates, with different numbers of lines and fields in each frame, has long been an issue facing the video industry. Today, the problem is compounded by the need to convert material from interlaced to progressive with a mix of SD and HD formats, each with its own color space.
Conversion systems now also must account for the embedded audio carried along with virtually all TV signals. Before audio was embedded in the digital signal, it remained the domain of audio engineers. However, integrating audio with digital video signals has made it hard to separate the two without introducing lip sync errors. Consequently, format conversion must be considered both an audio and video issue.
Standards conversion
A high-quality format and standards conversion solution is valuable whether implemented in a facility that performs live event transmission or mastering and duplication. Within the live transmission chain, there is no time for manual intervention, and the standards converter must be able to provide a high-quality picture, regardless of the circumstances. Any errors introduced during conversion will find their way into the end product. The quality requirements of a system used in post-production and mastering/duplication facilities are high as well. Clients often want to send the finished program to multiple markets, which requires the facility to convert material to the appropriate standards, corresponding frame and line rate, scan, and so on. The quality of the standards converter affects the end product's quality and the project's cost-effectiveness.
A less expensive converter with many operational modes may be attractive. However, the time and labor spent fixing the end product may quickly erode any cost savings. Manual intervention and quality assurance are costly, so conversion must be performed right the first time. It is therefore essential that the converter have only one operational mode, i.e., “On.” A robust standards converter that provides highly automated operation and sophisticated image and audio processing is usually a better choice.
Image processing tech
When purchasing a standards or format converter, it's important to know what type of underlying image processing technology is being used. There are several approaches to deinterlacing and frame rate conversion.
Early conversion solutions relied on a linear filter, combined in some cases with motion adaption technology to try to adapt how the output image behaves based on the presence of motion. The most basic linear deinterlacing solution filters out the odd or the even fields, reducing the image resolution by half.
Click image to enlarge
Adding a degree of sophistication to this process, motion adaptive techniques maintain both fields in those parts of the picture where there is no motion, but blend half the resolution in those parts of the picture with movement. Like linear techniques, this process leads to image softness, blurring and apparent motion judder.
Motion estimation provides an alternative method for converting interlaced video to progressive. There are three primary technologies used to estimate movement: block matching, gradient techniques and phase correlation.
Block matching and gradient techniques are the easiest to implement in hardware and software-based solutions. However, both share one detrimental side effect. Their performance is influenced negatively by changes in noise and luminance levels. Noise is present in all signals, and luminance changes are nearly as common as movement in TV programming.
One example of luminance level changes is a football kicked in late afternoon through sun and shadows across the field. Luminance levels change significantly as the ball transitions from sunlight to shade. Another example is models walking on a catwalk as flash photos are taken. These types of scenes can dramatically upset both block matching and gradient techniques.
Phase correlation
An alternative is phase correlation, which is based on the principle that the displacement in the time domain relates to the phase shift in the frequency domain. As a frequency domain technique, phase correlation is free from the effects of luminance and noise and thus offers good immunity to changes in both light and noise.
Implementing phase-correlated motion estimation requires sophisticated algorithms and a high degree of processing power. Within phase correlation, a Fourier transform breaks down the video into a series of sine waves. The phase for each of these sine waves is thus provided, and motion can be measured with the phase information available from successive images. Spectral analysis of two successive fields and subsequent subtraction of individual phase components yields phase differences that, when subjected to a reversed Fourier transform, provide a correlation surface with peaks corresponding to the motion between successive images.
By using multiple stages, it's possible to derive a motion vector to sub-pixel resolution. This level of accuracy provides for a system that is most able to recreate the movement and changes in a given scene.
During deinterlacing, motion compensation moves different parts of the picture to the correct point in time, recreating the missing lines from the interlaced scan so that each field is presented in full resolution. The same technique can be used to perform accurate frame-rate standards conversion, as both processes must be able to recreate the position of images at any point in time. When converting video from 50Hz to 60Hz, motion compensation can be used to replace 50 frames of video with 60 frames of video.
A common misconception is that 10 frames are either added or subtracted to achieve the correct frame rate. Actually, every frame in the converted output is synthesized from scratch. Because each frame-rate standard samples different points in time within the same one-second interval, phase-correlation technology measures the measure motion between two inputs that straddle the desired output field and then scales the motion vectors accordingly. As a result, an entirely new set of frames is generated accurately.
Processing challenges
In a motion-compensated standards converter, the interfield interpolation axis is not aligned at the time axis in the presence of motion. (See Figure 1.) In practice, the interpolation axis is skewed by using the motion vectors to shift parts of the source fields. The displacement is measured in pixels, and the value is divided into the integer part (the nearest whole number of pixels) and the fractional part (the subpixel shift). Pixels from input fields are stored in RAM, which the interpolator addresses to obtain input for filtering.
The integer part of the impulse response shift is added to the RAM address so that the pixels from the input field appear to have been shifted. The vertical shift changes the row address, and the horizontal shift changes the column address. Address mapping moves the image with pixel accuracy. This is followed by using the subpixel shift to control the phase of the interpolator. Combining address mapping and interpolation in this way allows image areas to be shifted by large distances with exceptional accuracy.
What makes the quality of motion estimation so critical is the behavior of outputs when the converter makes a mistake in creating an intermediate field. Say a camera pan causes a person to move from left to right; the motion compensation system must locate that person in a point in time never actually captured by the camera. If the technology moves the head and body differently, the human eye and brain will know at some level that something's not right. Thus a minute error in motion estimation, even for one output picture, is enough to create a significant disturbance for the viewer.
Fast and complex motion can be a challenge for conversion; however, speed can help mask the effects of poorly performed motion compensation. (See Figure 2 on page 34.)
| Want to use this article? Click here for options! |  |
