Digital video archiving
Jul 20, 2008 8:00 AM
As the amount of digital video increases, the need for a long-term storage solution becomes crucial to maintaining a station’s video assets. In the past, the videotapes were stored, hopefully in a climate-controlled room. But the difficulty in logging, researching and finding the tape, not to mention loading it onto a VTR and then shuttling to the clip needed, was often more trouble than the clip was worth. And even if the tape were found, it was very likely that it had not aged well — the magnetic oxide could be flaking off or foreign material may have caused the tape to stick to itself; there could be excessive dropouts or there may not be a VTR left that can actually play the tape. Unless it was a major network, archived videotapes probably stayed on the shelves.
Today, all of that has changed. With digital media, all archival footage is available 24/7 at the click of a mouse, or at least it can be if you build and maintain a storage system that is designed to expand. Currently, all online and nearline storage is hard disk-based, and archive storage is thought to be tape-based, but this is changing. New technologies are providing innovative ways to store and retrieve digital assets.
Archive or backupFirst, it’s important to be clear about what “backing up” and “archiving” means. To back up data means to make a copy of it in case the original is corrupted or lost. This is a temporary situation in that the data is continually changing, and backups of the data need to be constantly made to keep up with these changes.
When data is archived, it’s in its final form and will not be changed again. A copy is being made to keep for a long time (years, decades, centuries), and the original can be safely erased from the local hard disks.
Because of this, archives are much more important than backup because they hold the only copy of the data — if the archive is lost, the data is totally lost. With backup copies, the entire purpose is to have several copies spanning several days at least.
Tape, disk or opticalToday, there are more options for archive storage than ever before. The standard for archiving for many years has been data tape storage. Today’s newest data tapes can hold up to 1.6TB of compressed data; this equates to a cost of less than 10 cents per gigabyte, and some sellers warrant their data tapes for 30 years. The drawbacks to data tapes are the inherent risks of moving a thin magnetic tape at high speeds past tape heads with the possibility of tape damage and loss of the archive data. On the plus side, tape-based archives are a very mature technology with a long track record. Today, it is the most widely used archive system in the world.
Although read/write optical disks have been around for many years, today’s Blu-ray Discs offer much higher capacities than before. Blu-ray Discs now offer up to 50GB of storage. At current prices, Blu-ray storage costs about $1 per gigabyte. And even though the cost is much higher, the overhead to start and maintain an optical disk-based archive system is much lower, because you can copy files onto desktop Blu-ray burners. But for a greater volume of data, there are jukebox-type systems that will store up to 280TB on 50GB Blu-ray Discs.
Pioneer recently announced that it has successfully produced 400GB Blu-ray Discs using 16 layers with a capacity of 25GB for each layer. If applied to the above jukebox system, its capacity would increase to 2240TB, or 2.2PB.
Other archive systems use hard disk drives in RAID systems because they have high capacity and a known history. The cost of using hard disks comes to about $3 per gigabyte. To maintain these systems, the hard drives must be changed out every three years.
Disk-based systems must also be backed up to data tapes, which can include the archive data as well as active data. Transferring archive data to the tapes along with active data increases the number of data tapes required as well as the time it takes to transfer and cost. In a true archive system, only the active data would be backed up to data tapes, while the archive data remained on the archive media.
In the best of all possible worlds, permanent data would be written to an archive system once and then erased from all active data storage. All active data and only active data would be backed up frequently, providing several layers of recovery copies. All the storage systems would never lose that data and would never fail. But in the real world, cost is a major factor in how data is stored, backed up and archived. The lower the per-gigabyte cost, the better — talking in terms of hundreds or even thousands of terabytes.
There are many factors to consider when evaluating storage systems, some of these include:
- Power consumption: With rising energy costs, the electricity used to keep disks spinning adds up. Gigabytes stored per kilowatt consumed is something to keep in mind.
- Transfer speeds: Many archive systems are slow when it comes to transferring data, but speed becomes more important with backup systems because they transfer multiple terabytes of data on a regular basis. The faster a backup can be performed, the more secure the data is.
- Media longevity and equipment lifecycle: How long will the archive media last? Currently, 30 years is the longest any media is warranted for. How long will the storage system be supported and how will you migrate to a new storage system in the future?
These are all factors that normally would be the concern of the IT manager in a large company, but now TV engineers need to address these issues as more television becomes digital. Do we want to view and resuse today’s programs in 50 years?
Programs from the beginning of television broadcasting are only viewable today because the original kinescopes were transferred to 2in quad videotape, then to 1in type C videotape, then DigiBeta and now to a computer file residing on a RAID system. That was roughly 50 years and four migrations to keep our history available and viewable. What will happen in the next 50 years as technology evolves more rapidly and new systems are introduced and replaced at an even greater pace?
| Want to use this article? Click here for options! |  |
