Compression for digital cinema cameras, part 2

sony-s-log-graph-205be11fi2.jpg

sony-s-log-graph-205be11fi20.jpg

Most broadcast HD camcorders use straightforward DVC, MPEG-2 or MPEG-4 (H.264/AVC) compression. This orderly world of intraframe and interframe codecs is becoming more complex as digital cinema tools are adopted for high-end broadcast productions. In part 1 of this series (April 2012), we examined the processed category: compressed and uncompressed.

In part 2, we will focus on the technologies involved in recording motion pictures. The unprocessed category represents cameras that do not employ de-Bayering. (See Figure 1.) These cameras record RAW images. Think of RAW as a digital negative.

Figure 1. Storage options for motion pictures

The majority of the cameras we will look at use a Bayer filter. This requires post-production software perform a de-Bayer before an image is obtained. A Bayer-filtered sensor has only half the number of green samples per row and half the number of green samples per column as it has rows and columns. One of the functions of the de-Bayer process is to re-create as much missing green information as possible. The amount that can be recovered can be called “de-Bayer efficiency.” A working value for a high-quality process is 78 percent.

For a single-sensor camera to provide luminance resolution equivalent to that from a three-chip camcorder, the image to be de-Bayered must be equal to, or larger than, 3.4MP (HD) or 13.6MP (4K2K).

In the traditional compression world, the maximum recorded bit depth is 10 bits. Unfortunately, 10-bit linear values are not capable of carrying the dynamic range obtained by today's CMOS sensors. These sensors output samples with 12 or more bits. One solution: Use a codec such as AVC-Intra Class 4:4:4 that can record 12-bit data. Another solution is to record logarithmic data. (See Figure 2.) An 800 percent to 1000 percent greater dynamic range is made possible by outputting, for example, 10-bit log values from 16-bit samples. Figure 1 highlights those cameras that record linear data and those that record log (nonlinear) data.

Unprocessed/compressed — linear

The RED EPIC captures sequential RGB data from its 5K MYSTERIUM X sensor. (See Figure 3.) Data are sent into two paths. One path de-Bayers the data, thereby converting RAW sensor data to 2048 × 1080-pixel 12-bit RGB data. These data may be modified by the operator using ISO, white balance and other RGB color space adjustments. They are then scaled and gamma corrected to provide 4:2:2 10-bit YCbCr or 4:4:4 8-bit RGB monitor output. Operator adjustments make no changes to the recorded image.

Wavelet compression is applied to the RGB data sent down the second path. RED's “visually lossless” compression scheme is called REDCODE. Compressed image data and adjustment metadata are written to an SSD-based REDMAG drive.

A Sony F65 camera has a 20MP sensor with a 1.9:1 aspect ratio that matches the 4K DCI projection standard (4096 × 2160). Effective sensor resolution is about 18.3MP.

The 18.3 million sequential RGB samples from the sensor are 16-bit values. At 24fps, the sensor's output data rate is approximately 7Gb/s. At 60fps, the data rate is approximately 17.6Gb/s. Lossless compression is applied in-camera to drop the data rate by 3.2x to 2.2Gb/s (24fps) or 5.5Gb/s (60fps). These 16-bit data, called F65RAW, are sent via a fiber-optic connection to a docked Sony SR-R4 field recorder. F65RAW data are written to SRMemory cards.

From the 18.3 million digital values, post-production software can generate 8K (UHDTV), 4K2K, Quad HD, 2K and full HD. The latter three formats are super-sampled. However, before editing, F65RAW must first be uncompressed and demosaiced to a frame of RGB values.

To facilitate broadcast production, an F65 can output 10- or 12-bit S-log RGB data. These 1920 × 1080 data are obtained from a built-in downconverter that takes in 18.3 million pixels and outputs HD. When these F65 data are fed to an SR-R4, they are recorded using the MPEG-4 Studio Profile (SStP) codec. This codec uses an intraframe compression algorithm that has three bit rates: 220Mb/s (SR-SQ Lite), 440Mb/s (SR-SQ) and 880Mb/s (SR-HQ). Compressed data are written to SRMemory cards.

Unprocessed/uncompressed — linear

The Phantom Flex is capable of frame rates up to 1455fps using a one microsecond global shutter with a resolution of 2560 × 1600 pixels. The sensor outputs sequential 12-bit RGB data. These 12-bit RAW data are written using Vision Research proprietary uncompressed “cine” files to internal RAM. When the RAM is full, data are offloaded to a computer.

The WEISSCAM HS-2 has a 2048 × 1536 Super35 CMOS sensor with a global shutter that can shoot 2K at 1500fps and HD at 2000fps. The HS-2 can output an RGB stream that employs Weisscam's “RAW IN HD SDI” mapping that transports 12-bit uncompressed RAW files. These data can be recorded by a WEISSCAM DIGIMAG DM-2 recorder.

Unprocessed/compressed — nonlinear

The Sony F3 records 35Mb/s 4:2:0 MPEG-2 to SxS cards. The F3's secondary path outputs 10-bit S-log RGB data. (See Figure 4.) This path does not allow image adjustments. (Only white balance can be specified.) Output of RGB values is via dual-link HD-SDI ports. A Sony SR-R1 field recorder stores the 10-bit S-log RGB data. The SR-R1 records SR-SQ Lite, SR-SQ and SR-HQ compressed data to SRMemory cards.

The Panavision Genesis has a 12.4MP (5760 × 2160 pixels) Super35 CCD. (See Figure 5.) You'll note that a Bayer pattern is not used. Instead, the sensor is binned to a 6.2MP, 14-bit per-color signal that is converted to Panalog data.

Panalog, developed by Panavision, transforms the internal 14-bit linear digital signal into a 10-bit quasi-log signal. RGB output is by dual-link HD-SDI connectors. A Sony SR-R1 field recorder can record these data using SStP compression.

The Silicon Imaging SI-2K and SI-2K MINI camcorders have a 16:9, 2/3in CMOS sensor with 2048 × 1152-pixel resolution. Analog samples are converted to digital values by a 12-bit A/D. RAW 10-bit log data are then processed using the visually lossless CineForm wavelet-based codec. Direct-to-edit AVI and QuickTime files compatible with CineForm Prospect 2K are then written to an internal interchangeable hard drive.

The SI-2K is under the control of built-in software called SiliconDVR. IRIDAS Speedgrade is one of the functions of SiliconDVR. Speedgrade enables a camera operator to set metadata for a clip. When editing, after de-loging and de-Bayering image data, metadata can be applied in real time to a clip.

Unprocessed/uncompressed — nonlinear

As shown by Figure 4, a Sony F3 outputs 10-bit S-log RGB via a dual-link HD-SDI connection. When an F3 is connected to an SR-R1 field recorder that has an SRK-R311 option, data can be recorded without compression and written to SRMemory cards.

The ARRI ALEXA has a single 3392 × 2200 photosite CMOS sensor that has 2880 × 1620 effective 16-bit pixels. For maximum quality, 2880 × 1620 pixels are output as ARRIRAW. ARRIRAW is log 12-bit uncompressed RGB data. ALEXA's output port is called T-link. It sends data as packets. The T-link physical connection is a dual-link HD-SDI connection. ARRIRAW can be recorded using a Codex ARRIRAW Recorder.

To support broadcast production, HD (1920 × 1080) video is obtained using 1.5x oversampling from the 2880 × 1620 pixels. After de-Bayering and conversion, YCrCb values are compressed using ProRes 422 Proxy, ProRes 422 LT, ProRes 422 or ProRes 422 HQ. Compressed 10-bit data are recorded to SxS cards inserted into the camcorder.

Post-production workflows for digital negative cameras will be more complex than those for ENG/EFP camcorders using traditional compression systems. (See Figure 6.) Digital negative workflows require specialized knowledge and can be time-consuming. Nevertheless, these camcorders and their workflows provide a true 35mm film digital replacement system for the production of episodic television and commercials.

Figure 6. Compression options for motion pictures

Steve Mullen is the owner of DVC. He can be reached via his website at http://home.mindspring.com/~d-v-c.