Motion estimation
Aug 1, 2007 12:00 PM, BY JOE ZALLER
Reducing artifacts delivers a cleaner picture and simplifies compression and decompression.
Deinterlacing
Figure 4. Phase correlation block diagram
Click image to enlarge.
Many viewers watch television on flat-screen displays that use progressive scan technology. Most video, however, is still acquired and transmitted using the interlaced scan technology created for CRTs in the 1930s. The ability to deinterlace video with minimal artifacts is a key video processing technique. Beyond its significance as a discrete process, deinterlacing is also a key step in standards conversion and SD-to-HD upconversion.
Interlacing video can be understood as a primitive form of compression. For a 480-line NTSC display, it sends 240 odd lines and then 240 even lines 16 milliseconds later. Thanks to persistence of vision, this is sufficient to create the illusion of moving pictures without requiring all 480 lines to be sent with each field.
The problem in converting interlaced video to a progressive scan, in which the 480 lines are drawn on the screen sequentially from top to bottom, is that the interlaced fields were sampled at different points in time. Unless the picture is entirely static, there has been motion between the two fields.
One way to convert interlaced video to progressive format is to merely filter out either all the odd or all the even fields, but this would reduce the resolution of the image by half. Motion-adaptive processing techniques preserve full resolution for the static areas of the picture, but still throw away half the resolution for the moving parts of the image.
Only motion compensation can convert interlaced video to progressive without losing any resolution on both the stationary and moving parts of the picture. By calculating motion vectors, it is able to recreate the missing lines from an interlaced scan. Thus, it restores each field to its full resolution.
However, there are several ways to perform motion compensation, and not all deliver the same level of vector reliability and precision. The highest level of vector reliability and precision is obtained by phase correlation, which analyzes incoming images and shifts the pixels in each video field so that they line up perfectly, even for complex scenes where there is fast motion, fine graphics or mixed film and video material.
Standards conversion and upconversion
Standards conversion is another genre of video processing that benefits from motion compensation and phase correlation. The benefits are clear if we consider that the component parts of the standards conversion process are deinterlacing, frame rate conversion and scaling.
The frame-rate conversion process benefits from the accurate and reliable motion vectors of phase-correlation motion compensation for the same reason that deinterlacing benefits. The difference, in the case of 50Hz to 60Hz conversion, is that the technique replaces 50 frames of video with 60 frames of video, each of which are sampling different points in time within the same one-second interval.
Consider, then, the challenges of performing this kind of processing on a pixel-by-pixel basis between formats of HD video, where the bandwidth is five times higher than SD, and there are five times the number of vectors to be calculated with five times the precision. The HD-to-HD conversion process involves a minimum of 16-bit resolution with 3Gb/s data rates throughout, and more advanced picture building techniques are required to deal with the issues associated with revealed and obscured areas of the image.
At the same time, the screens on which HD video is watched tend to be larger, making artifacts much more obvious than on the typical SD screen. For such applications, phase-correlation motion compensation technology is capable of delivering outputs that are artifact-free, low in noise, and high in detail and fidelity.
The future
Motion compensation technology was once only available in high-end broadcast products, but over the last decade, it has begun to trickle down into higher-volume commodity products. This technology migration will continue, and we can expect that high-end algorithms will find their way into display devices and consumer-grade software.
Certainly motion compensation will continue to be a key technology block in the future, even as it advances from today's 2-D applications to 3-D motion vectors that describe not only horizontal and vertical motion but also give an accurate measurement of depth, or Z-plane motion. With this added dimension, and with further increases in picture resolution to come, the possibilities for advanced high-efficiency algorithms will be virtually unlimited.
Joe Zaller is vice president of strategic marketing for Snell & Wilcox.
| Want to use this article? Click here for options! |  |
