HD editing storage
Jun 1, 2007 12:00 PM, BY SARAH STANFIELD
Bruce Motyer, Technicolor lead editor, uses Facilis Technology’s TerraBlock Manager to control access to uncompressed HD video files on the Avid DS Nitris.
Studios and post-production facilities are finding it increasingly commonplace to edit several terabytes of HD material each day. If a storage solution doesn't have the bandwidth and capacity to handle such file loads, chances are good that there will be bottlenecking and delays, because the server simply cannot keep up with the demands placed on it. Multiple HD editing systems require more.
If you find this situation at your facility, it's probably a good idea to start looking into one of the many shared storage solutions available for HD editing applications. While this may seem like a daunting proposition, take heart: Building an HD storage system, or even adding a new system to your existing storage platform, is not as complex as it seems.
A good storage vendor can work with you to create a solution that fits your facility's needs. Your job is to know enough about your storage requirements to present an accurate picture of your situation to the vendor.
Capacity calculations
The first order of business is to determine how much storage your facility will actually need. Which shared storage system you choose will ultimately depend on how efficiently and quickly it can manage the millions of bits of media data required for real-time HD editing projects, so it's important to know how much capacity it must handle. A general rule of thumb is to assume you will need about five times the amount of storage for HD as you currently use for SD.
To come up with this number, first determine how many workstations in your facility will need to be connected to the storage system. Take into account any new systems that you may add as part of the storage system installation. Then think about the specific projects typically performed on an application basis on each station:
Which applications require (or will require) the management of uncompressed versus compressed HD files?
Which ones perform the most complex functions (such as special effects), require editors to add handles or perform cross fades, or may need extra bandwidth and capacity for other items?
Which ones require the playback of more than one track on a particular timeline?
A thorough understanding of these issues — down to the application level — will give you the best estimate of the amount of HD storage capacity you will need.
As you look at your editing systems, remember to take into account bandwidth as well as capacity. Real-time HD playback requires a lot of bandwidth. For example, about 1TB of HD storage capacity for every hour on the timeline requires 277MB/s bandwidth just to be able to play an HD file back in real time. Most facilities have more than one editing application trying to access the storage system at the same time, so it may be necessary to multiply this number by the number of editing systems in the facility.
Design decisions
After determining the overall bandwidth and capacity required by the facility, the next step is to research the particular type of storage architecture that works best for your facility. The four main types of storage architecture available today include three storage area networks (SANs) — server-assisted, direct-to-storage and server-direct — and a server-based network attached storage (NAS). Of these four storage architectures, you will likely want to choose a server-assisted or server-direct SAN, as these systems are generally the best at handling large amounts of HD material.
Server-assisted SANs
Server-assisted SAN solutions have been on the market for quite some time. They can support many clients and are cross platform, meaning different clients can write to the system at the same time. Sharing is done on the file level, with the workstation client requesting access to files from the server's processing system.
While the fact that the system only processes requests and not actual data makes it fast, bottlenecks can arise when too many workstations are trying to read and write files through the system. This can also be a problem when several applications requiring high bandwidth are making requests. In addition, the file system is custom, so it is necessary to install software on each client to make its operating systems compatible with the server.
Server-assisted SANs are a good choice for facilities that handle fast-turnaround media, such as live feeds for broadcast news or reality programming. They generally work well with media traveling in and out of the facility quickly. They aren't the best solution for facilities specializing in high-end editing projects, such as episodic television or film work. This kind of work generally requires the media to be available on the server for a longer period of time, and bottlenecks may become a problem. It is possible to work on HD projects with these systems, but it often requires a complex IT infrastructure.
Server-direct SANs
Server-direct SANs are the latest incarnation of shared storage architecture. Sharing is done by the block level, and the storage is made available to clients via a virtual volume scheme, allowing the client to format pools of storage from different portions of the physical volumes. Each virtual volume will appear on all the clients connected to the SAN, allowing for collaborative editing. In addition, the virtual volumes can be created on a project basis and deleted without affecting another portion of the storage pool. The file system is native, so no client software is required to make client operating systems compatible with the SAN.
Facilis Technology’s TerraBlock 24D can scale to 18TB in a single server. Technicolor uses multiple servers for more than 50TB of online storage.
The main limitation of this system is that it doesn't have multiwrite, so users cannot write to the same virtual volumes from different locations. However, this aspect can likely be worked around through good project management.
Server-direct SANs are especially adept at handling episodic television or film work, as the virtual volume feature allows the media to stay on the server for long periods of time with minimal bottlenecks. Not surprisingly, if most of your projects involve quick turnaround edit times, the fact that the system cannot support multiwrite might mean it is not the right solution for you.
Vender venture
The next step is talking with specific storage vendors. Look for companies that specialize in media (as opposed to data) storage systems. This may seem like an obvious point, but there are some vendors that may try to sell you a system that is better optimized for data storage. While these systems can be extremely fast (and therefore appear to be ideal for quickly moving media around your facility), the physical hardware has been designed for data — not media backup.
The vendor should also be familiar with the specific editing applications you need to tie to the SAN. The vendor doesn't need to know the ins and outs of these applications but should have good understanding of how they interface and talk with the SAN. While many SAN systems on the market are plug and play, you will likely need to tweak the system so it interfaces smoothly with all the editing systems connected to it. It's also important for the vendor to understand the editing applications if you plan to add its solution to an existing SAN or NAS system, as the vendor may need to add another layer of software to allow the various systems to interface with one another.
Hardware honing
In terms of the actual hardware for your SAN, most vendors offer expensive Fibre Channel drives or a more cost-effective alternative called Serial ATA (SATA). SATA disks have become increasingly sophisticated over the years and now offer almost the same amount of speed as Fibre Channel and can pack a lot of capacity into a small space. It's not uncommon today to see SATA-based servers that have a capacity of 18TB in 5RUs.
Need to know
In the end, choosing an HD editing storage platform comes down to truly understanding the unique needs of your facility. If you know the bandwidth and capacity needed overall, have a basic understanding of the workings of server-assisted and server-direct SAN systems, and know the alternatives to Fibre Channel disk drives, you will be in a solid position for choosing the right vendor for you facility. Good luck, and remember to ask about the warranty!
Sarah Stanfield is a technical writer and the author of Broadcast Engineering's Mobile TV Update e-newsletter.
| Want to use this article? Click here for options! |  |
