Format scan and conversion
Dec 1, 2007 12:00 PM, BY DAVID TASKER
The original purpose of the standards converter was to address the incompatibilities between the world's multiple television standards. Dealing with standards featuring different frame and field rates, with different numbers of lines and fields in each frame, has long been an issue facing the video industry. Today, that problem is compounded by the need to convert material from interlaced to progressive with a mix of SD and HD formats, each with its own color space.
In addition to addressing these image-related factors, conversion systems now also must account for the embedded audio carried along with virtually all TV signals. Before audio was embedded in the digital signal, it remained the domain of audio engineers. However, integrating audio with digital video signals has made it hard to separate the two without the danger of introducing lip-sync errors. Consequently, format conversion must be considered both an audio and video issue.
Standards conversion workflows
An investment in a high-quality format and standards conversion solution is valuable whether implemented in a facility that performs live event transmission or mastering and duplication. Within the live transmission chain, there is no time for manual intervention, and the standards converter must be able to provide a high-quality picture, regardless of the circumstances. Any errors introduced during conversion will find their way into the end product.
The quality requirements of a system used in post-production and mastering and duplication facilities are high as well. Clients often want to send the finished program to multiple markets, and this requires the facility to convert material to the appropriate standards, corresponding frame and line rate, scan (interlaced or progressive), and so on. In this type of workflow, the quality of the standards converter affects not only the quality of the end product, but also the cost-effectiveness of the project.
Sometimes saving money on a less-expensive converter with many operational modes may be appealing. However, the time and labor spent fixing the end product may quickly erode any cost savings. Manual intervention and quality assurance are costly, so conversion must be performed right the first time. It is therefore essential that the converter have only one operational mode, i.e., “On.” Post and duplication facilities cannot afford rejection of the converted product, as it could damage both the company's bottom line and its reputation. A robust standards converter that provides highly automated operation and sophisticated image and audio processing is usually a better choice.
Image processing technologies
The most important thing to know in purchasing a standards or format converter is the type of underlying image processing technologies being used. There are several approaches to deinterlacing and frame rate conversion. Let's look at them along with their respective strengths and weaknesses.
Early conversion solutions relied on a linear filter, combined in some cases with motion adaption technology to try to adapt how the output image behaves based on the presence of motion. The most basic linear deinterlacing solution simply filters out all of the odd or all of the even fields, effectively reducing the image resolution by half.
Adding a degree of sophistication to this process, motion adaptive techniques maintain both fields in those parts of the picture where there is no motion, but they blend half the resolution in those parts of the picture with movement. Like linear techniques, this process leads to image softness, blurring and apparent motion judder.
Motion estimation provides an alternative method for converting interlaced video to progressive. There are three primary technologies used to estimate movement: block matching, gradient techniques and phase correlation.
Block matching and gradient techniques are the easiest to implement in hardware and software-based solutions. However, both share one detrimental side effect. The performance of these techniques is influenced negatively by changes in noise and luminance levels. Noise is present in all signals, of course, and luminance changes are nearly as common as is movement in television programming.
One example of luminance level changes is a football kicked in late afternoon through both sun and shadows across the field. Luminance levels change significantly as the ball transitions from sunlight to shade. Other examples include models walking on a catwalk as flash photos are taken and actors walking the red carpet as paparazzi take pictures. These types of scenes can dramatically upset both block matching and gradient techniques.
Phase-correlated motion estimation
An alternative to block matching and gradient techniques is phase correlation, which is based on the principle that the displacement in the time domain relates to the phase shift in the frequency domain. As a frequency domain technique, phase correlation is free from the effects of luminance and noise and thus offers good immunity to changes in both light and noise.
Implementing phase-correlated motion estimation requires sophisticated algorithms and a high degree of processing power. Within phase correlation, a Fourier transform breaks down the video into a series of sine waves. The phase for each of these sine waves is thus provided, and motion can be measured with the phase information available from successive images. Spectral analysis of two successive fields and subsequent subtraction of individual phase components yields phase differences that, when subjected to a reversed Fourier transform, provide a correlation surface with peaks corresponding to the motion between successive images.
By using multiple stages, it's possible to derive a motion vector to sub-pixel resolution. This level of accuracy provides for a system that is most able to recreate the movement and changes in a given scene.
The same technique can be used to perform accurate frame-rate standards conversion, as both processes must be able to recreate the position of images at any point in time. When converting video from 50Hz to 60Hz, motion compensation can be used to replace 50 frames of video with 60 frames of video.
A common misconception regarding standards conversion is that 10 frames are either added or subtracted to achieve the correct frame rate. Not so. Every frame in the converted output is synthesized from scratch, so to speak. Because each frame-rate standard samples different points in time within the same one-second interval, phase correlation technology measures the motion between two inputs that straddle the desired output field and then scales the motion vectors accordingly. As a result, an entirely new set of frames is generated accurately.
| Want to use this article? Click here for options! |  |
