 The ability to store massive amounts of content and transfer it into and out of storage libraries has enabled the move toward a tapeless environment. |
The IT world is rapidly changing the landscape of broadcasting. The fundamental change began with the ability to convert analog video into a well-organized digital package of bits and bytes. Though digital formats have existed for years (D1, D2, D3, Digital Betacam, etc), only recently have video packages become storable, manageable, and transferable files. This translates into much greater efficiency, both in the speeds at which video can be moved and the ways in which it can be manipulated.
The basic difference between data in the broadcast world and data in the traditional IT world is that this data is a broadcast engineer's lifeline. It is their product. It's the news packages, spots, promotions and programming, all of which depend on modern technology to deliver business value. If an e-mail server becomes disabled, a company and its employees are inconvenienced. If a video server becomes disabled, things could get very ugly, very fast!
Another basic distinction between video and data is the enormous size of video files. An uncompressed two-hour SD file could easily chew up several hundred gigabytes of storage. Storing files in uncompressed form requires massive storage capacity and creates significant challenges when it comes to transporting the content.
Before digital compression, video could only be transferred in real time. And while video could be routed around a facility digitally using serial digital interface, a one-hour program would still take an hour to transfer. Along came digital compression, and with it, greater efficiencies in transfer rates and storage.
Video post production was revolutionized in the late 1980s with the introduction of nonlinear editing — the ability to manipulate video content digitally with the kinds of tools that had previously changed the face of word processing and document publishing. Then, in the 1990s came the introduction of broadcast video servers — high-performance, highly reliable computers taking the place of VTRs. Both systems used digitally compressed video along with their own proprietary file systems, storage systems and video formats. While they increased productivity exponentially within their operation, rarely did they communicate with one another. The problem became interoperability: what to do with the data once the proprietary storage devices were full. Typically, the content was played back out to videotape where it sat on a shelf until needed. This meant the tape would need to be re-ingested or re-encoded back into the broadcast server if it was to be aired again or sent back to the NLE system if it needed to be edited or re-purposed.
The issue then became, how does a broadcaster manage the growing storage needs of the new IT-based content it is accumulating? The answer came from traditional IT storage vendors, who pioneered the introduction of large robotically automated, tape-based libraries capable of holding thousands of hours of content.
In 1986, one of these libraries could hold 120 terabytes via 6000 tape cartridge slots. Today, an automated tape library can hold 2.6 petabytes via 6500 slots, with throughput over 500Mb/s, enabling the storage of more than 100,000 hours of video at a hefty 50Mb/s in a single library.
 Figure 1. Digital content archive architectural overview. Click here to see an enlarged diagram. |
Additionally, files can now be transferred at speeds of up to 10x real time. New tape drive technology will continue to push the envelope for density and throughput. This will have a greater impact as more broadcasters move to HD environments where storage and bandwidth requirements typically increase by a factor of six.
The process of taking an IT storage device and plugging it into a video server, however, was fraught with challenges, including proprietary file systems, the need to communicate with automation systems and the need to transfer content from library to server and back again on demand. The solution was borne from a niche group of software vendors who developed middleware known as archive managers.
Archive managers solved this problem in several ways. First, they integrated with broadcast automation systems so that they could take commands from those systems when a request for content was made. As play/programming lists were created, automation systems made commands as content was deleted and online disk storage space became available. In addition, as new content was ingested into a video server, the automation system would be programmed to send a duplicate copy to the archive for future reference. This insured that aired content could simply be deleted with the master residing in the tape archive.
As archive systems developed, intermediate disk buffers would be added between the broadcast servers or NLE systems and the tape libraries. The purpose was to facilitate hierarchical storage management, now known as content lifecycle management. Content lifecycle management works by storing content needed for play out within X number of days on the server. Content frequently re-aired is stored on intermediate, less expensive disk. And all remaining content is housed in the tape library representing the ultimate content repository.
Extending this model even further, archive managers can now manage a remote disaster recovery (DR) site where a second tape library sits with a duplicate set of all video files. If an accident were to occur in the primary location, the archive manager can pull content from the DR library and move it across a WAN to the primary broadcast facility for play out or re-editing.
The ability to store massive amounts of content and transfer it into and out of storage libraries has enabled the move toward a tapeless environment. Because digital tape libraries and disk subsystems are agnostic in terms of the video formats supported, multiple formats can be stored and recalled from the same storage device, creating immediate efficiencies. Over time, the cost of maintaining and repairing VTRs for each tape format will diminish as more content is ingested and digitally archived.
There is also a favorable trade-off between storing content on expensive videotape versus much more economic and faster data tape. Today, an LTO3 tape can hold approximately 30 hours of DV 25 video at an estimated street cost of about $5 per hour, or roughly 70 hours of MPEG-2 10Mb/s at a cost of about $2 per hour. In addition, while there are certainly significant cost savings in media, there are also clear savings in terms of the overall hardware footprint. Imagine how much floor space a 200,000-hour DV 25 news library would take up versus a 60sq-ft data tape library.
There are also clear productivity gains to be realized, such as no longer having to re-ingest content that has been played out to tape because the disk on the server is full. Also, content often needs to be shared between servers playing out different channels, which means continually re-ingesting the same files. A centralized digital content archive solves these problems by maintaining content in digital form, eliminating the need for re-ingestion.
Some of these issues have been addressed by the broadcast vendors who have steadily added more disk storage to their servers and created shared file systems and SANs for their post and news operations. Even so, digital content creation and distribution has outpaced that fix and, therefore, the need for tape libraries — large and small — continues to grow. An important factor is the cost ratio of disk vs. tape: 100 to 1. That's a lot of money.
In 2005, it is now common to see digital infrastructures being built from the ground up for television stations as well as larger cable and broadcast networks. Central to these facilities is their respective digital storage and archiving infrastructures. As HD continues to proliferate, this need will increase. So too will the need for broadcast engineers and IT professionals to peacefully co-exist and become more immersed in each other's pursuits. There is a growing need for professionals that understand IT as well as broadcast engineering — both in the technical sense, but just as importantly, in the business sense. Clearly, the need to understand and embrace data storage and digital content management is integral to this new paradigm.
Ron Quartararo is the Broadcast/Media & Entertainment manager for StorageTek.