Applied Technology: 3DV Systems' Zcam

3DV Systems' Zcam

By Dr. Giora Yahav and Dr.Gavriel Iddan

By integrating its 3-D imaging concept with a conventional studio camera, 3DV Systems created Zcam, an RGBD studio system that can separate the background and add/subtract image segments in real time.

Sometimes a new piece of equipment can change the way business is done, literally, overnight. Until now, chroma keying has always been required for online manipulation of video images. Not any more. Recently, 3DV Systems introduced Zcam, a new approach to keying and segmentation that’s based on real-time video-rate range-mapping of the whole field of view (FOV). The RGBD studio system allows objects to be attached to range layers and segmented in an image, eliminating the need for a special chroma studio and enabling real-time creation of a variety of effects.

True metric 3-D imaging – attaching a numerical value to the range of every pixel of an image – requires imaging and computation far more complex than simple 3-D stereo imaging. The methods used for this type of imaging are generally based on either triangulation or time of flight (TOF). The system is based on TOF. By integrating its 3-D imaging concept with a conventional studio camera, the company created an RGBD studio system that can separate the background and add/subtract image segments in real time. (The “D” in RGBD stands for distance and represents the fourth gray-level image.)

Camera concept

TOF was chosen because it provides direct numerical range information, which is essential for real-time video-rate operation. In addition, TOF can achieve the required speed because it simultaneously addresses the whole field of view.

To envision this concept, imagine a system that includes a pulsed light source and a gated CCD camera. The light source puts out a train of short light pulses, typically 10 nsec. Each pulse becomes a “light wall” that moves forward and illuminates the field of view so that the field of illumination (FOI) is congruent with the FOV.

When it contacts objects in the FOI, the light wall captures imprints of the objects and is then reflected back towards the camera. As it enters the camera, a fast shutter blocks the trailing half of the light wall. The energy detected by each camera pixel corresponds inversely to the distance that generates a gated image (GI), so the light reflected from a near object is captured completely while the reflection from a distant object appears too late to be captured. Instead, the CCD camera captures a gray-level image where dark objects are distant while bright objects are close.

To complete the process, a normalization procedure is used to compensate for non-uniformity in the illumination and variations in the coefficient of reflectivity. During this procedure, an additional un-gated image (UGI) is captured. The normalized image is generated by the pixel-by-pixel ratio: R(i,j) = GI(i,j) / UGI(i,j).

Because timing of the illumination and the gating process can be operator-controlled, the operator can locate the range window to correspond to both the talent location and the distance of talent from the camera.

But a TV studio camera must also capture the highest quality color video (RGB) in addition to range image. The RGBD camera incorporates both color and range simultaneously. (As mentioned earlier, the “D” in RGBD represents a fourth gray-level image corresponding to range, as compared to traditional studio cameras that provide RGB only.) The four images overlap, allowing the device to achieve high quality range keying without modifying existing studio facilities. In fact, any existing standard studio-quality RGB camera can be used with Zcam after a simple adjustment procedure to incorporate it.

During the design process, the company solved several technological challenges. The camera required several components that were not available anywhere. The company designed, patented and fabricated its own solid-state large-aperture image gate, for use instead of a traditional gated intensifier tube. The company also integrated the color and range channels into the same taking lens. This added unique precision zooming capabilities.