Employing Data Center Bridging in media networks
Jan 1, 2010 12:00 PM, By Luc Andries
Figure 1. IEEE 802.3x pause frame mechanism
Select figure to enlarge.
As more broadcasters transition to file-based media production systems, Internet Protocol (IP) has become the transport technology of choice. Applying an IT-based infrastructure to media introduces benefits to media workflow systems, but also creates new requirements that must be addressed. One area that demands particular attention is storage, as media applications pose unique storage requirements that differ from classical IT solutions.
An IP-based media storage architecture must provide robust parallel throughput and high capacity, scalability, redundancy and availability. The nature of a file-based media workflow means it must operate in a multiuser environment and provide simultaneous access to files (leading, by definition, to a random access pattern on the underlying file system and storage architecture). Because of the continuous nature of media applications, the throughput of each I/O operation on media storage must also be guaranteed under all circumstances. While a slow e-mail is still an e-mail, a slow video is no longer video.
To meet these requirements, each technology layer of a media storage environment must perform optimally and provide guaranteed throughput, including the storage network. Today, many file-based media storage environments use InfiniBand (IB) storage network interfaces due to its high link bandwidth (16Gb/s throughput for double data rate IB) and low cost per port. Now, a new technology, Data Center Bridging, has emerged that can offer performance advantages over IB. Our lab tested DCB against IB in a typical media network and found that DCB provides a compelling media storage solution.
Creating a lossless network
Figure 2. GPFS media storage WARP cluster
Select figure to enlarge.
A stringent requirement for any storage network technology is to provide a lossless environment. IB and DCB accomplish this in different ways. IB uses a buffer-to-buffer credit mechanism to avoid frame loss. Credits of available buffers are continuously exchanged between ports on the same link. When no buffer credits are available, no packets are transmitted until the network processes its congestion and buffers become available again. Hence, the receiver never needs to drop frames.
DCB provides a lossless network through the use of a pause frame mechanism defined in the IEEE 802.3x standard. Conventional Ethernet does not keep track of the buffer availability on the receiving end of a link and assumes by default that buffers are available, creating a risk of buffer overflow on the receiver. With the pause mechanism, a receiver that notices that its buffers are being filled above a certain threshold can send a pause frame back to the source, telling it to hold all transmissions for a certain amount of time. (See Figure 1.) Once the buffers have returned below the threshold, the source can resume transmissions.
In a media cluster environment, with its simple well-defined topology, the 802.3 pause mechanism should demonstrate an observable behavior close to the lossless buffer-to-buffer credit mechanism of IB.
Running the test
We based the test architecture on the General Parallel File System (GPFS) media storage cluster from IBM, one of the most powerful media file systems available. A GPFS cluster based on the network-attached node (NAN) model consists of storage cluster nodes and network-attached cluster nodes. The storage servers are directly connected to the storage, whether locally attached or via a storage-area network (SAN) architecture. NAN nodes are via a cluster network connected to all storage nodes, but are not directly attached to the underlying storage. In this architecture, each storage node is the access point or primary server for part of the total storage. The NAN node stripes its data requests over all storage nodes, thereby aggregating the available bandwidth of each individual storage node and connected storage subsystems.
IB-based WARP cluster
Figure 3. DCB-based GPFS media storage WARP cluster
Select figure to enlarge.
The IB cluster architecture, named Workhorse Application Raw Power (WARP) media storage cluster, displays a many-to-one traffic pattern. When a NAN node reads from the storage nodes, all the storage nodes respond at the same time back to the NAN node with large bursts of traffic. If, on the other hand, multiple NAN nodes write data to the storage, the receiving storage nodes are simultaneously addressed by the bursts of all the writing NAN nodes. Both cases result in heavy oversubscription of the cluster network.
The InfiniBand stack is extremely efficient for Linux-based servers, reaching the full physical limits of the underlying bus technology. The processing of the protocol stack is fully offloaded in the IB host channel adapter network cards. Even remote direct memory access is fully supported and exploited. This leads to a powerful cluster architecture, extremely well adapted for file-based media production environments. (See Figure 2.)
| Want to use this article? Click here for options! |  |
