Managing storage in file-based workflows
Aug 1, 2010 12:00 PM, By Paul Turner
A complete file-based workflow should include concepts of centralized storage, clip management and overall process management.
A media company's decision to implement a tapeless, file-based workflow marks the start of significant work for the facility's engineering department. A failure to define this undertaking in concrete terms at the outset can make it difficult or impossible for engineering to implement such a system successfully. Any individual workflow incorporates many discrete processes, and the goal of an efficient file-based workflow can only be realized when all these processes have been optimized to the greatest extent possible. To realize a complete file-based workflow, the system designer must include the concepts of centralized storage, clip management and overall process management, considering the impact of each of these on the overall system design and complexity.
Infrastructure considerations
The Omneon platform at KCSM-TV in San Mateo, CA, enables the college-owned station to realize cost and time savings of tapeless production and to migrate smoothly to HD.
The first step in migrating any facility to a file-based workflow is to replace the physical transport infrastructure, and this can be a relatively straightforward process of replacing the original baseband (and therefore real-time) interconnection of processors with IT-based connectivity schemes. IT-based backbones can offer significantly higher aggregate bandwidth between media processors, and that increase in performance, coupled with transportation of media as compressed data, can lead to an improvement in throughput. But simply replacing an existing video-centric routing scheme with an IT-centric routing scheme in and of itself cannot be classed as creating a file-based workflow.
Problems arise if the individual media processors in the facility (logging systems, NLEs, graphics systems, QC systems, transcoders, audio processors, etc.) still operate as processing “islands,” each with its own media storage subsystem that holds the media while the processor acts upon it. Reliance on islands of storage makes it likely that, at any point in time, a broadcast facility is storing a number of copies of any single piece of media — probably in multiple formats. In this case, the broadcaster needs a way to track and manage that material in order to maximize the efficiency of the operation.
A classic example in which islands of storage present a problem might be when a particular processor, such as an NLE, requires 40GB of local storage to hold a piece of media that it needs, but only 30GB are available. Though another NLE in the facility may have spare capacity available, the first system can't use that storage, as it's dedicated to the second NLE, which may or may not be from the same manufacturer. The “island” model would suggest that the broadcaster solve this problem by buying more storage for the first NLE system. A far more cost-effective solution is to share storage across this nonhomogeneous edit environment and allocate it to systems as needed.
To realize a complete file-based workflow, the designer must include the concepts of centralized storage, clip management and overall process management as part of system design. The rest of this article will examine the impact of each of these on the overall system design and complexity.
Centralizing media storage
A centralized storage architecture can offer huge benefits to any media enterprise. As a single entity, a centralized storage system offers significantly reduced management requirements, as storage can be allocated to external devices as needed and modified as the broadcaster's needs change. Such storage systems also can simplify expansion of storage capacity, as any added storage becomes available to any external device rather than being dedicated to a single function. (See Figure 1 on page 44.) When designed with an open file system, the storage system can allow multiple nonhomogeneous systems to access stored material.
Figure 1. Central shared storage architecture
Select figure to enlarge.
For the central storage model to be effective, system design must address a number of additional parameters. First, as a central resource, the storage system cannot simply be taken out of operation for a software update, storage expansion or similar maintenance. If this were to happen, the facility would grind to a halt. Storage systems are available that offer “hot” upgrade capabilities, but the system designer should take this real-world issue into consideration and exercise care when choosing a storage product; otherwise, a backup storage infrastructure must be deployed to keep the facility operating while the central storage is offline. This scenario also requires that source material and finished material be migrated to and from the central storage in a managed fashion before and after the downtime occurs.
Another consideration when adopting a centralized storage system is bandwidth scaling capability. As the demand for available storage increases, there is an associated increase in the number of clients who wish to access that storage. In addition, new business requirements often dictate that higher bit rate files be processed by the facility, which means that larger files are being stored on the system itself and that increased bandwidth to the client will be needed to guarantee acceptable operation. Thus, the storage system needs to be able to grow in available bandwidth in a managed manner, or the inevitable result is that the system disintegrates into a number of islands of storage, each providing capacity and bandwidth to a constrained number of clients.
If the storage will be used as a replacement to local storage on NLEs, then experience shows that one of the crucial parameters for the central storage is access latency. Operators are used to a certain level of service in terms of immediacy of content — as provided in the past by their local media storage — and their expectation of a new central storage scheme, regardless of its scale, will be that it consistently provide the same or better performance.
Even in a facility that employs central storage, some islands of storage will likely remain. Ingest and playout servers, for example, may continue to operate on their own storage systems for security purposes. Consequently, the broadcaster still will need a solution for managing and monitoring the multiple copies of content residing on multiple file systems, if only to ensure that a particular clip can be appropriately archived or all copies can be deleted from the system as required.
Managing stored material
A crucial factor in any true file-based workflow is management of the material on the storage systems. There are many levels of content management, and each level offers a balance between ease of implementation (cost) and complexity of solution.
| Want to use this article? Click here for options! |  |
