Scaling beyond SAN
Nov 1, 2009 12:00 PM, By David Austerberry
Many newer storage architectures are more cost-effective than NAS and SAN.
Much of the evolution of file storage in the broadcast environment has taken place on an ad-hoc basis. The island of production — graphics, news and editing — built storage systems to meet its own special requirements. (See Figure 1.) The original storage that could meet the needs of video production, direct-attached storage (DAS), was later supplanted by a mix of network-attached storage (NAS), a specialized file server and storage-area networks (SAN). Each has its advantages and shortcomings.
Some of the rationale broadcasters adopted when buying storage systems rested on the assumption that media files were just works in progress. The real archive was videotape, and it was used to move content between the production islands. In the last few years, the broadcast business has radically changed, and many of the changes are affecting storage requirements.
Videotape technology is reaching its end of life phase. Now that cameras can output video as files as well as streams, there are many alternate ways to store video. Optical disks, solid-state memory and hard drives are replacing videotape for acquisition. The recent decreases in price of fiber bandwidth mean that to transfer content between the production processes video networks can replace a courier moving a tape.
It's not only the technology that has changed; broadcasters must distribute more programming to more distribution outlets and in more formats. This is driving down revenue per channel, so broadcasters are forced to look for more efficient processes. These can be summed up as file-based operations — leveraging the computing, storage and networking of the IT sector.
As broadcasters replace their videotape libraries with file storage systems, the search is on for storage that meets the needs of the media industry. Broadcasters are more cost-conscious than many traditional users of enterprise storage (government, healthcare, oil and gas), yet the requirements can stretch technology. Media files are large; an hour of uncompressed 1080p50 video is nearly a terabyte. Real-time transfer rates for such a file are 375MB/s and require network technologies like 10GbE.
The older storage architectures, NAS and SAN, could handle SD files, but the move to 3Gb/s video means that broadcasters must look to more recent storage configurations to meet their needs. To provide scope on some of the requirements, a typical 15,000 rpm Fibre Channel hard drive has a sustained data transfer rate around 1.2Gb/s, which is below the real-time video transfer rate. A compositing workstation running eight video layers of uncompressed 1080p video would need a 24Gb/s connection to the storage. Of course, compression is normally used, with codecs like DNxHD and ProRes 422, and drives are striped together, but this indicates how HD video is demanding of the storage and networks.
Apart from the basic technical needs, some of the requirements broadcasters will look for in a storage system include:
low energy use;
low latency file transfers;
security, possibly with encryption;
ease of maintenance;
50-100 year archive life;
compatibility with legacy systems;
automated migration to new media; and
DR facilities.
It should also have a low cost of ownership and high reliability. Unlike many enterprise users, broadcasters use a mix of operating systems for their client workstations — Windows, Mac and Linux — which means flexibility with the network file systems.
Many buzzwords are flying around the storage world: clusters, grid, scale-out, virtualization and cloud, to name just a few. Cloud storage can virtualize a cluster of storage for broadcasters that want to scale out its capacity. Many of these newer architectures are well suited to the needs of broadcasters and can prove more cost-effective than the traditional NAS and SAN.
Network attached storage
NAS has been a popular storage appliance, partly because it is a simple concept. It can be considered as a specialized file server, a familiar concept with any IT engineer. The problem with the NAS filer is that it doesn't scale well. The network interface is a bottleneck and limits the rate files can be served. If more NAS systems are added to a network to increase capacity, the problems are space and asset management. A user will see a number of mounted drives, but where do they store the files? If one appliance fills up, there is no automatic mechanism to use available space on another appliance. The consequence is inefficient use of the available disk storage.
What is needed is an abstraction of the physical storage from the user. That way the user sees a pool of all the available storage. The complexity of drive letters and worrying about tape backup can be hidden from users; they see a virtual data store.
Scale-out NAS
The various drawbacks of NAS have led storage vendors to look for an architecture that allows the NAS to scale out to provide large storage systems and the virtualization that hides the complexities of running enterprise storage systems.
Scale-out NAS can be a cluster of file stores. As files are written to one of the cluster nodes, a management network synchronously replicates files across the cluster. Multiple nodes add capacity to the system and provide higher throughput for read-write operations. The replication provides resilience. The file management network allows the cluster to be presented a single namespace, one drive letter, simplifying the storage mount for users.
Storage hierarchy
Storage comes in a hierarchy of performance and cost, from Fibre Channel disks and solid-state drives at the high end, down to offline data tapes. One size does not fit all, so to cover all the performance demands of broadcast and to optimize cost, hierarchical storage management (HSM) mixes storage technology. A typical broadcast system may use SAS drives for real-time video serving, SATA drives for low-cost nearline storage, and a data tape library or virtual tape technologies for the archive.
| Want to use this article? Click here for options! |  |
