Digital conversion
Dec 1, 2006 12:00 PM, BY ALDO CUGNINI
Aspects of format conversion have been known and appreciated for some time, due to the frequent need to convert between different video standards. Many of the same techniques that were developed for analog systems apply in the digital world. However, additional techniques are needed, some of which are not always appreciated. To understand the basis of digital conversion techniques, let's look at the analog situation first.
Analog conversion
When converting between two analog television systems, several elements must be changed, including frame rate, line rate, scan method and signal encoding. In effect, the first three elements are changed by an interrelated form of sample rate conversion, which was the subject of last month's Transition to Digital column. This element trio essentially defines the pixel rate of the system. Therefore, in order to convert between two different standards, it's necessary to do the appropriate sample rate conversion both spatially and temporally, meaning within a field or frame and between the fields or frames.
Figure 1. Frame X shows proper temporal interpolation of the movement of the object from Frame A to Frame B.
Mathematically, the appropriate interpolations or decimations need to be done in both the spatial and temporal directions. In practice, the situation is more complex because of the use of interlaced scan.
For simplicity, let's first consider a fictional case, with two systems at 30Hz and 60Hz frame rates and both using progressive scanning. Let's also assume that both systems have the same vertical line rate. Converting from the 30Hz system to the 60Hz requires interpolation between the 30Hz frames and the creation of new frames. In principle, this would seem to be the same as a change in spatial resolution, which would involve spatial interpolation between adjacent spatial pixels, in order to create new pixels. However, the time dimension must account for objects in the picture that can move from one frame to the next. This creates a new requirement. Now, it's necessary to predict the motion of these objects to faithfully reproduce their motion in a new frame.
Intermediate images
By illustration, let's assume an object is moving horizontally, such that it appears at the locations shown in Figure 1 at the times A and B. (The translation is exaggerated here for clarity.)
Figure 2. Frame X shows the result of poor temporal interpolation from Frame A to Frame B.
Logically, an interpolator should produce an image as shown in the intermediate frame X. However, a straight pixel-by-pixel temporal interpolation by averaging the successive frames would actually produce the result shown in Figure 2. Obviously, this is not the correct way to create the new frame.
In fact, using a more sophisticated filter to reconstruct the intermediate image, such as a (sin x)/x, creates an even worse situation. Such a filter requires many taps, or coefficients, in the frame-to-frame time direction. The result of such a filter is that the moving objects would become smeared over the same number of frames.
Motion estimation
An intelligent motion estimator is needed to determine the motion of objects within the picture and to create new pixels based on this motion. Before the advent of high-speed digital processing, this was impossible to do. And format conversion inevitably resulted in serious conversion artifacts when rendering sequences of moving objects.
Extending the principle to conversions between field rates of 50Hz and 60Hz (or 59.94Hz), the difference is that it may be necessary to alter each frame to smoothly transition from groups of five frames to groups of six. Conversions in the reverse direction can also involve motion estimation, as the mere dropping of one out of every six frames would result in jerky video.
Interlaced scan
Adding interlaced scan to the situation further complicates matters. Interlace can be thought of as a vertical-temporal image sampling that alternates phase every field. (See Figure 3.) By further extension of the motion estimation technique, this sampling can be taken into account.
The difference is that objects having a vertical component to their translation should be processed using a different algorithm than objects with a purely horizontal motion. De-interlacing, that is conversion from interlace to progressive, often entails maintaining the same image resolution. Fully generalized conversion, on the other hand, adds the element of spatial resolution to the process. Finally, the different analog standards require a conversion of the signal encoding techniques, such as luminance levels, chrominance encoding and blanking signals.
| Want to use this article? Click here for options! |  |
