Employing Data Center Bridging in media networks
Jan 1, 2010 12:00 PM, By Luc Andries
DCB-based WARP cluster
Table 1. IB-based WARP cluster throughput results.
Select table to enlarge.
One potential issue of media environments that places constraints on IB-based architectures is that many media client applications require a Microsoft Windows operating system. This is the case for both Windows applications that have to run on the NAN cluster nodes and applications that require a mount of the central file system via the CIFS protocol. Recently, IBM added a GPFS-on-Windows client to its NAN node configuration. This allows a Microsoft Windows 2003 server to participate as a NAN node in the GPFS cluster. The state-of-the-art IB stack for Windows machines is, however, much lower-performing than the Linux version. The cluster protocol stack has to fall back to using IP-over-Infiniband (IPoIB) without any offloading, because not all GPFS commands are yet supported in the native IB stack for Windows. This decreases the performance of the cluster network by a factor of five.
Because GPFS is agnostic to the underlying network technology, the GPFS WARP cluster can be designed with DCB technology replacing IB as the cluster network. (See Figure 3.) This should be especially beneficial in the Microsoft Windows NAN node environment. For our test case, we created a DCB architecture using the Cisco Nexus 5000 Series switch, a widely deployed 10Gb/s Ethernet data center platform that supports 802.3 PAUSE frames. Our tests showed that a DCB-based GPFS WARP cluster is capable of making use of the full bandwidth provided by the 10Gb/s DCB platform.
The results
Table 2. Single-connected DCB-based WARP cluster throughput results.
Select table to enlarge.
We configured the IB-based system to use the “dd” application as stream generator and used DDR IB in an active/active setup using verbs. The storage throughput limitation for the system was ~800MB/s per storage system, and the cluster consisted of three NAN nodes and four storage nodes. As a baseline reference, we performed throughput benchmark tests on the IB-based clusters with both Linux and Windows NAN nodes. The tests included:
- single stream throughput from one NAN node;
- multiple streams from one NAN node (for a more even saturation of the link bandwidth); and
- multiple streams simultaneously from all NAN nodes.
Table 1 shows the results.
For the next phase, we replaced IB with 10Gb/s DCB in the storage cluster network. The IEEE 802.3x pause mechanism was activated, both on the Cisco Nexus 5000 switch and on the converged network adapters (CNA) to create a lossless Ethernet. The DCB-based WARP cluster tests were executed with Nehalem-based servers as NAN nodes, as they were able to fully saturate the 10Gb/s link.
In the first DCB setup, all servers were connected with a single 10Gb/s link to the Cisco Nexus 5000 switch, compared with the double active/active setup of IB. In a second test, dual links were used. The traffic between different server pairs was routed over different interfaces. We performed the same benchmark tests as were used for the IB-based system. The results for the single-connected and double-connected systems are shown in Tables 2 and 3.
Table 3. Double-connected DCB-based WARP cluster throughput results.
Select table to enlarge.
These results show that the DCB system outperforms the IB cluster on these benchmarks. A single NAN DCB-based cluster outperforms the IB-based cluster by 663 percent, and three NAN DCB-based clusters outperform the IB-based cluster by 384 percent. The performance gain for the Windows setup is remarkable.
Ultimately, DCB is an effective solution in real-world media environments. The Windows performance of a DCB-based GPFS media storage WARP cluster opens up opportunities to connect many Windows-based editing stations to a generic media storage cluster. With this architecture, it is easy to envision a high-resolution HD post-production editing platform that scales well beyond the state-of-the-art solutions presently offered by media vendors.
Luc Andries is ICT-architect at VRT-medialab.
| Want to use this article? Click here for options! |  |
