Facing media traffic challenges
Feb 1, 2010 12:00 PM, By Luc Andries, MSC
The way the IT world characterizes and defines IP traffic must change to accommodate the demands of today's media.
Figure 1. Large media files act more like trains than cars in IP network traffic, causing congestion.
Select figure to enlarge.
As media companies transition to file-based production environments, many have experienced problems translating media applications to an IP environment. They find that IP networks do not behave as expected. Throughput decreases and becomes unpredictable, transfers are lost, and conventional network monitoring tools cannot explain why these issues occur.
The reason for this mysterious behavior is that media traffic is intrinsically different from IT traffic, especially at small-time scales. Traditional tools used to measure and characterize IT traffic no longer apply. A new mode of thinking is required.
Our laboratory sought to describe this unique behavior and identify the specific requirements for an IP network to support media file transfers. Ultimately, we created an IP network infrastructure that was more than capable of handling media traffic flows.
Comparing IT and media traffic
Figure 2. This test configuration used Avid editing clients providing multiple streams of video, all connected to a generic media storage cluster architecture.
Select figure to enlarge.
IT traffic generally consists of short messages or small files, such as those generated by SAP or e-mail, which use the IP network for only relatively brief time periods. Transferspeed is not critical, and packet loss and resulting retransmission are acceptable. Media traffic, however, generally consists of large files (several gigabytes), which typically use the link for a longer time period and almost constantly try to use 100 percent of the available bandwidth. The longer this period, the more “bursty” the traffic becomes.
When two traffic streams share the same link, the unique nature of media traffic exacerbates the problem of bandwidth competition between concurrent transfers, leading to packet loss. Any required data retransmissions decrease the overall efficiency of the transfers drastically and, if sustained, can lead to complete transfer interruption. This occurs even when the network has been designed (at least on the macroscopic scale) with sufficient bandwidth to accommodate both streams.
To understand why this happens, consider an analogy of traffic traveling along a two-lane highway. (Figure 1.) Imagine two clients running standard IT services, sending traffic at a speed of 400Mb/s to a common file server. Since each car in the stream is relatively small, the two lanes of IT traffic can merge (i.e., when they reach the network switch) and efficiently combine into a single lane. The server receives the traffic at 800Mb/s without much delay. Bandwidth or throughput adds up linearly in most cases.
Now examine the right portion of Figure 1; it illustrates what happens when large media files are being transported over the same network. Keeping with the transportation analogy, we see that the large media files actually behave more like trains approaching a junction than cars (i.e., long, continuous bursts of traffic arriving back to back). Two clients sending large files to the same server at 400Mb/s no longer manage to get all the traffic through to the server without interference. If both trains (large media files) arrive at the (switch) junction simultaneously, they crash into each other; all traffic is stopped. Consequently, the receiving media server does not attain an aggregated throughput of 800Mb/s, but much less. The bottom line from this example is that data/throughput can no longer be added linearly.
Unfortunately, these effects are not typically visible when applying classical network monitoring tools. Such tools are geared toward IT traffic and typically measure throughput by averaging over relatively long time intervals. Building networks for media traffic requires fundamentally different tools that can look at the network on a time scale several orders of magnitudes smaller. At this scale, concepts such as average network throughput become meaningless. In these cases, a network link is loaded with a packet, or it is idle. There is no such thing as bandwidth percentage anymore.
Testing HD over IP networks
Figure 3. A look at how media traffic behaves on different time scales
Select figure to enlarge.
To demonstrate these effects, the VRT-medialab analyzed the IP network traffic generated by an editing client connected to a media storage cluster on different time scales. As a challenging test case, we used a set of Avid Media Composer editing clients using multiple streams of HD video, all connected to a generic media storage cluster architecture like that shown in Figure 2. The network used a Workhorse Application Raw Power (WARP) media storage cluster from VRT-medialab as a generic storage platform. Cisco Nexus 5000 switches with lossless 10GE and Data Center Bridging functionality were used in the cluster network, providing a generic media storage solution with ample throughput for HD editing.
Several Avid Media composer editing clients were connected via a Cisco Nexus 7000 switch to the storage cluster using both 1Gb and 10Gb links. IP media traffic was transported via the SMB/CIFS (Server Message Block/Common Internet File System) protocol between the server and the client.
Analyzing media traffic
The analysis consisted of playing a single DN×HD 145Mb/s HD video stream with a frame rate of 29.97 frames per second over a single 1Gb connection. Figure 3 illustrates the traffic moving between the storage and client.
The left side of the figure displays the macroscopic time scale of seconds and measures an average bandwidth of around 150Mb/s. This corresponds nicely with the compression bandwidth of the DNxHD 145Mb/s codec. The measured throughput is slightly higher than the codec specs, due to the header and protocol overhead in the TCP/IP packets. The first 1.6 seconds, the client is prefetching a number of video frames into its buffers to overcome the eventual jitter of the storage and network. During this period, the measured macroscopic bandwidth is around 580Mb/s, 4X the codec spec, indicating the client is reading four times faster than the playout speed.
| Want to use this article? Click here for options! |  |
