New digital cinema tools enable more options for high-end shooting.
From the introduction of the DV format, most camcorders have used DVC, MPEG-2 or MPEG-4 (H.264/AVC) compression. These codecs can be placed into one of two categories: intraframe and interframe. Color sampling is 4:1:1, 4:2:0 or 4:2:2. Data depth is 8 bits or 10 bits. This orderly world of compression is being replaced by one with far greater variety as high-end shooters adopt digital cinema tools.
In this new world, video is treated as “motion pictures.” There are two fundamentally different ways photo cameras record pictures: JPEG and RAW. (See Figure 1.) JPEG compression we know as the basis of motion-JPEG. It is a DCT-based, lossy, intraframe compression technique. RGB data from a sensor are de-Bayered and then spatially compressed. RAW, simply put, is RGB data from the sensor without de-Bayering. With no compression applied, the file for each image will be large. Therefore, visually lossless compression may be applied to reduce file size.
| Processed (de-Bayered) | Unprocessed (RAW) | |
|---|---|---|
| Compressed | JPEG | |
| Uncompressed | RAW |
The advantage of RAW is twofold. First, a photo editor works directly with sensor RGB data, which enables a wider range of image adjustments. Second, there are no JPEG compression artifacts. The disadvantage of RAW is that image manipulation is performed more slowly. Slow manipulation is not only the result of working with a large frame of data; each image must be software de-Bayered before processing.
We will look at recording motion pictures in four ways. (See Figure 2.) While the categories remain the same as Figure 1, more detail is introduced. Current digital motion picture cameras employ all four of the possible variations presented in Figure 2. In this article, part 1, we will examine the processed category. Both compressed and uncompressed recording will be covered.
| Processed | Unprocessed (RAW) | |
|---|---|---|
| Compressed | Processed/compressed | Unprocessed/compressed |
| Uncompressed | Processed/uncompressed | Unprocessed/uncompressed |
Consider a simplified three-chip camcorder. (See Figure 3.) Three sensors output either analog or digital information. In the former case, A/D converters are part of a camera's first image processing stage. In the latter case, digital data from each sensor are ready to be processed. Either way, a sample has a bit depth of 10 bits to 16 bits.
The first processing stage converts RGB data streams to YCrCb (luminance, red chroma component, blue chroma component) data. Processing is typically performed using 14- to 16-bit DSP. First, the luminance (Y) signal is computed. Second, two color “difference” components (Cr = Y - R and Cb = Y - B) are computed. The luminance signal is obtained by adding each pixel's red, green and blue samples together. Each sample is multiplied by a coefficient that represents the amount the color signal contributes to the Rec. 709 luminance signal (Y' = 0.213R + 0.715G + 0.072B).
Color sampling from three-chip camcorders is 4:4:4. These YCrCb data may be filtered to 4:2:2 prior to second-stage processing that provides image adjustments: black level, black compress, black stretch, white level, gamma, saturation, hue and sharpness/detail.
The sharpness/detail adjustment controls how much the video signal is boosted in the mid-range to prevent the loss of fine detail. (See Figure 4.) Boosted mid-frequencies emphasize edges.
Unfortunately, hard edges define objects that, when moving, increase our perception of judder. The red and blue curves represent, respectively, a minimum and maximum sharpness/detail adjustment.
Even at a normal (black) setting, mid-frequencies have a moderately large peak. Negative detail correction, as offered by some camcorders, attenuates mid-frequencies. This correction reduces the perception of judder while not diminishing high-frequency fine detail.
An alternate to way to prevent images from suffering from a lack of fine detail is to capture and process very high-definition images. This is accomplished by 4K2K cameras. This makes them ideal for both digital cinema and high-quality broadcast production.
When a codec supports only 4:2:0 sampling, YCrCb data are further filtered. And, depending on the compression scheme, 10-bit to 16-bit data are converted to 8-bit or 10-bit data. The specifications for eight common video codecs and one very-high-data-rate codec, HDCAM SR (SRMASTER), vary. (See Figure 5.)
| Format | Bit depth | Resolution | Chroma sampling | Bit rate | Compression |
|---|---|---|---|---|---|
| HDV | 8 bits | 1280 × 720, 1440 × 1080 | 4:2:0 | 18Mb/s-25Mb/s | Interframe |
| H.264/AVC | 8 bits | 1280 × 720, 1440 × 1080, 1920 × 1080 | 4:2:0 | 18Mb/s-36Mb/s | Interframe |
| AVCHD | 8 bits | 1280 × 720, 1440 × 1080, 1920 × 1080 | 4:2:0 | 18Mb/s-28Mb/s | Interframe |
| XDCAM EX | 8 bits | 1280 × 720, 1440 × 1080, 1920 × 1080 | 4:2:0 | 25Mb/s-35Mb/s | Interframe |
| MPEG-2 XDCAM 422 | 8 bits | 1280 × 720, 1920 × 1080 | 4:2:2 | 50Mb/s | Interframe |
| DVCPRO HD | 8 bits | 960 × 720, 1280 × 1080, 1440 × 1080 | 4:2:2 | 100Mb/s | Intraframe |
| HDCAM | 8 bits | 1440 × 1080 | 3:1:1 | 144Mb/s | Intraframe |
| AVC-Intra | 10 bits | 1280 × 720, 1440 × 1080, 1920 × 1080 | 4:2:2 | 50Mb/s or 100Mb/s | Intraframe |
| HDCAM SR | 10 bits | 1920 × 1080 | 4:2:2 or 4:4:4 | 440Mb/s or 880Mb/s | Intraframe |
HD camcorders use a DVC, MPEG-2 or AVC-based compression. Compressed data are recorded to tape, disc or solid-state memory. Though most camcorders employ 4:2:2 processing with 10-bit data, the majority of their codecs employ only 8 bits and many provide only 4:2:0 color sampling. The solution, as shown in Figure 3, is an uncompressed digital output. YCrCb data are output via an HDMI or HD-SDI port. Typically, output is 10 bits with 4:2:2 color sampling. Data can be sent to an external recorder that has an Apple ProRes or Avid DNxHD hardware codec.
Consider a single-chip camcorder. (See Figure 6.) Sequential RGB data from the sensor are de-Bayered prior to YCrCb processing. For a single-sensor camcorder to provide luminance resolution equivalent to a three-chip camcorder, the image to be de-Bayered must be equal to, or larger than, 3.4MP (full HD) or 13.6MP (4K2K). The de-Bayer interpolation process can be nearest neighbor, linear, cubic, cubic spline, etc.
De-Bayering generates an equal size 4:4:4 color-sampled RGB frame. When the sensor is larger than the frame-size to be recorded, downscaling is performed. This can be done before or after conversion to 4:2:2 YCrCb. Image adjustments are YCrCb processed.
Long-GOP MPEG-2 or H.264/AVC encoding is typically used to record YCrCb data. When a codec supports only 4:2:0 sampling, YCrCb data are chroma filtered. Likewise, 10-bit to 16-bit data are converted to 8-bit or 10-bit data. YCrCb data can be output via an HDMI or HD-SDI port. (See Figure 6.) Typically, output is 10 bits with 4:2:2 color sampling. Data can be sent to an external digital recorder that has an Apple ProRes or Avid DNxHD hardware codec.
Next we'll look at a single-chip camcorder that employs “super-pixels.” (See Figure 7.) Sensor resolution is four times larger than the recorded image. Data from every four photosites are combined to create a super-pixel that has two green samples. This ratio nicely matches the requirements for the luminance signal: Y = 0.213R + 0.715G + 0.072B. No de-Bayering is required to obtain an RGB frame. Therefore, resolution matches the RGB frame size, and color sampling is 4:4:4. Data, however, are filtered to 4:2:2 prior to image adjustments. The YCrCb data are encoded using 8-bit 4:2:2 long-GOP MPEG-2.
YCrCb data can be output via an HD-SDI port. Output is 8 bits with 4:2:2 color sampling. Data can be sent to an external digital recorder that has an Apple ProRes or Avid DNxHD hardware codec.
To provide higher quality recordings, a camcorder can provide an uncompressed output. Shown in Figures 3, 6 and 7, an uncompressed YCrCb digital signal is output via an HDMI or HD-SDI port. The port is fed a digital signal after image adjustments have been made. Depending on a camcorder's processing circuitry, the 4:2:2 output may be 8 bits (125Mb/s) or 10 bits (166Mb/s). To record uncompressed data, the recorder must be able to store data at these rates.
The second part of this article will focus on the compression technologies involved in the recording of RAW motion pictures.
Steve Mullen is the owner of DVC. He can be reached via his website at http://home.mindspring.com/~d-v-c.