Analog video performance tests are usually carried out using static waveforms. Signal quality measurements are a reasonably good way to determine the picture quality. This is due to the fact that there is a strong correlation between the shape of the video test waveforms and the perceived picture quality with very few exceptions.
The reason signal quality measurements work with analog and full bandwidth digital systems is that uncompressed systems are linear. Signal quality testing does not work well with the compression/decompression processes. Traditional test signals are easily compressed with little distortion or loss.
Figure 1. Equipment evaluation test setup. Click here to see an enlarged diagram.
Due to the lack of distortion, these signals do not evaluate the compression/decompression process. For example, classical signal-to-noise ratio (SNR) measurements using a flatfield or a ramp are not a reliable measure of compressed picture quality. They are not constant and can give misleading results.
Video compression is a nonlinear process. The picture quality is a function of bit rate, picture complexity and encoding algorithm capabilities. MPEG-2 is essentially a lossy compression method. The compressor analyzes a video image, decides what is important information and discards the unimportant information.
Compression algorithms are very good but not perfect. Sometimes certain areas of the image are considered to be less important than others. If important information is discarded, this becomes evident after decompression. Typically, the background of the image, particularly the darker areas, are affected. These distortions have a noise-like appearance and can be categorized as:
Edge business: Distortion concentrated at edges of objects, moving artifacts or noise patterns superimposed over objects.
Mosquito noise: Edge business associated with movements.
Quantization noise: “Snow” or “salt & pepper” similar to random noise but not uniform across the image.
Figure 2. Bidirectional test setup with loop-through in Paris. Click here to see an enlarged diagram.
The PQR performance measurement compares the reconstructed image present at the output of the system with that present at the input of the system, pixel-by-pixel, and expresses the difference in numbers representing the deterioration perceived by the human vision system (HVS). The comparison is carried out by a computer that uses an algorithm simulating the HVS. The performance levels expressed in PQR on a scale of one to 10 are interpreted as follows:
PQR = 1: The picture degradation is hardly perceptible.
PQR = 3: The picture degradation is slightly visible.
Table 1. Test results of the MPEG codec. Click here to see an enlarged diagram.
The PSNR measurements are estimates of the quality of the reconstructed image compared to the original image and are expressed in dB. Acceptable PSNR figures vary between 20dB (acceptable) and 40dB (excellent).
Double-ended systems have access to the pre- and post-compression program material. By comparing the original image and the post-compression image or the level of pre- and post-compression impairment, an indication of picture quality can be gained. This obviously has the limitation of the need to have access to the original signal.
Various video test sequences are available on CD-ROMs. A sequence lasts five seconds, of which two seconds are used for analysis. I have been using three sequences, known respectively as “Diva,” “BBC” and “Mobile with calendar.” These sequences have different image details and movements complexity. “Diva” is the least stressful, and “Mobile with calendar” is the most stressful.
Table 2. Test results of the MPEG codec with the intercontinental loop. Click here to see an enlarged diagram.
Some time ago, I was involved in the specification and acceptance tests of a bidirectional (Canada>Europe> Canada) MPEG-2 system with input and output SDI 270Mb/s and a 4:2:2 compressed video bit-rate of 7.3Mb/s. The distribution network was fiber-optics, and the transport mechanism was ATM. I carried out several sets of tests using the three video sequences known as “Diva”, “BBC” and “Mobile with calendar.” I am presenting the results of two tests:
Typical measurement results
Test #2: A bidirectional (loop-through) test comprising the transatlantic bidirectional ATM transport with a loop-through in Paris. The test setup is shown in Figure 2 and the test results in Table 2.
Comparing the results in Table 1 with those in Table 2, you will notice an almost imperceptible picture degradation. This reinforced the expectation that under normal circumstances, the picture quality is essentially determined at the beginning (compressing) and the end (decompressing) of the transmission link. Given error-free transmission of the data signal, the picture quality remains unchanged over the entire path.
The compressed picture quality affects everyone in the broadcast chain. The most effective approach to assessing the performance is a combination of subjective and objective tests. The test setups and the video sequences used need to be consistent, and the measurement results should be referenced to an industry standard.
Broadcasters still have the problem of ensuring adequate picture quality through the transmission chain. This requires the maintenance of the highest standards, including the interoperability of concatenated compression/decompression systems using different technologies. An often neglected aspect is the statistical multiplexing of several programs, which generates a time-varying quality factor. As you undoubtedly noticed, we are not quite there yet.
Michael Robin, a fellow of the SMPTE and former engineer with the Canadian Broadcasting Corp.'s engineering headquarters, is an independent broadcast consultant located in Montreal, Canada. He is co-author of “Digital Television Fundamentals,” published by McGraw-Hill and translated into Chinese and Japanese.
Send questions and comments to: firstname.lastname@example.org