In his April 2006 “Computers & Networks” column, Brad Gilmer teased Broadcast Engineering readers about the future of 10 Gigabit Ethernet (10GigE) and its potential impact on the broadcast industry. The future is a lot closer than many in the industry think.
The 10GigE standard was adopted some four years ago. While it initially saw little application, the increasing demand for faster connections is moving it to the forefront.
For several years, there has been a discussion within the networking industry that 10GigE was going to sweep in and replace the current Gigabit Ethernet (GigE) infrastructure because of the need for ever increasing bandwidth.
While 10GigE offers huge performance improvements, this transition simply hasn't yet happened. The reason: As a backbone upgrade in the enterprise space, it is still too expensive compared to using GigE connectivity in the broadcast industry, where the pressure to support HD is being relaxed. Therefore, the resulting need to edit in true uncompressed HD format was not as time critical as initially expected in many facilities.
With the constant technological changes, it is often difficult to know exactly when something moves from being impractical to reasonable. An even worse situation may occur when a technology quickly becomes required and overdue, and a facility hasn't planned for its implementation.
With greater adoption of 10GigE occurring in other market segments, the price of 10GigE is trending lower. (See Figure 1.) With last year's introduction of the new CX4 copper interface for 10GigE, the cost of 10GigE will continue to decrease even faster.
Other factors also will contribute to the increasing market penetration of 10GigE, including new computers converting to higher performance PCI Express peripheral busses, which provide a full bandwidth solution for the 10GigE format. Also, there is growing interest in using 10GigE protocol in smaller LANs in combination with iSCSI, which will allow network attached storage (NAS) solutions to provide all editors quicker access to large video files than is presently available with Fibre Channel-based storage solutions.
With all of these different technologies leveraging the benefits of 10GigE, the price/performance crossover point will continue to be driven lower through 2009. These factors will make it easier for video equipment manufacturers to build HD products with both speed and large storage capabilities.
The power of 10GigE will begin to change the way we will want to move our data. The Ethernet network protocol has withstood the challenges of FDDI, ATM and every other network-based protocol.
In the networking industry, there is the saying “It's Ethernet or Ethernot, and Ethernet always wins.” Ethernet is the most commonly installed fabric in the industry. (See Figure 2.) Other fabric options that provide like performance levels are proprietary and are lower volume, which lead to higher per connection (or port) costs. Proprietary fabrics require specialized skill sets to install, operate and maintain, generating a higher cost of ownership.
Complete interoperability with all Ethernet vendors' solutions is an accepted fact, providing the customer with more flexibility in selecting and maintaining their data storage solutions. Ethernet network switches are much less expensive than their proprietary fabric counterparts. Being able to transfer files 20 times faster with 10GigE will also eliminate the need for sneakernet via FireWire for moving your data between various locations in-house, providing improved security of your data.
10GigE is a flexible fabric, capable of supporting multiple market segments with different configuration types. Because of its high performance and comparatively low cost, 10GigE is being used as high performance computing cluster (HPCC) fabric. Clusters are many systems rack-mounted to provide a compact physical footprint and allow many processors to work on the same problem concurrently.
Typically driven by price-performance metrics, which are critical in this application, 10GigE continues to enjoy market share improvement by driving for lower latency designs. 10GigE is also a viable fabric in server-to-server and server-to-storage network configurations. With 10GigE's inherit performance advantage over 1Gb, 2Gb or 4Gb storage solutions, 10GigE is capable of providing faster storage access. 10GigE fabric can improve server infrastructure costs by reducing the number of servers required to host the storage in typical SAN solutions compared to NAS configurations. 10GigE represents the next logical evolution for LANs as the need for ever increasing bandwidth from users will require the bandwidth improvements that 10GigE can provide over current GbE networks. (See Figure 3)
Installing a 10GigE fabric is not significantly different from the fabrics that are typically installed today. The first decision to be made is to determine the 10GigE fabric type — fiber or copper — that can properly support the site's requirements. There are tradeoffs to be made in fabric medium selection. While fiber will support distances of up to 300m between the switch and the computer systems, it carries a cost premium of 30 percent to 40 percent, depending upon the 10GigE vendor.
For smaller distance requirements, the 10GigE CX4 copper medium supports up to 15m between the switch and the system.
The next step is selecting the right 10GigE switch. Some switches will allow concurrent support of both 10GigE and GigE, which offers the flexibility of upgrading the network in sections. This would allow the initial implementation costs to be staggered over a longer time.
The routing of the 10GigE fiber cables is comparable to the routing of current Fibre Channel storage networks. The 10GigE CX4 cable is a little larger diameter than standard GigE cables. It is similar in routing to InfiniBand cabling, which for 15m runs requires that some thought be given to routing paths to ensure that bend radius requirements are adhered to.
The biggest advantage to the 10GigE fabric is its plug-and-play nature. Most computer operating systems have the necessary code embedded to support the fabric. Additionally, most 10GigE vendors have enhanced software available to allow the customer to realize the maximum performance possible from their products. 10GigE can be a subset of an overall GigE network. This allows a more staged migration for situations where 10GigE performance is needed in only a subsection of the network between the power users while leaving the rest of the current infrastructure intact.
| 2004 | 2005 | 2006 | 2007 | 2008 | 2009 | 2004-2009 CAGR (%) | |
|---|---|---|---|---|---|---|---|
| 10GigE | |||||||
| Revenue (SM) | 302 | 835 | 1536 | 1857 | 2300 | 3959 | 67.3 |
| Ports (000) | 41 | 212 | 628 | 1133 | 2002 | 5352 | 164.5 |
| ASP ($) | 7307 | 3945 | 2447 | 1640 | 1149 | 740 | -36.7 |
| 1000MB | |||||||
| Revenue (SM) | 7691 | 9569 | 11,611 | 13,095 | 13,857 | 12,837 | 10.8 |
| Ports (000) | 31,109 | 59,436 | 106,458 | 157,021 | 208,360 | 265,822 | 53.6 |
| ASP ($) | 247 | 161 | 109 | 83 | 67 | 48 | -27.9 |
| 100MB | |||||||
| Revenue (SM) | 6025 | 4728 | 3598 | 2712 | 1851 | 1100 | -28.8 |
| Ports (000) | 175,551 | 175,372 | 165,536 | 147,457 | 122,753 | 83,260 | -13.9 |
| ASP ($) | 34 | 27 | 22 | 18 | 15 | 13 | -17.4 |
Once a 10GigE fabric is up and running, there are additional steps that can improve a network's workflow. If the number of clients to the server exceeds two, consider configuring link aggregation into the server.
Link aggregation allows the server to support multiple Ethernet connections concurrently. For example, when moving large files that take several minutes to transfer, link aggregation allows the server to support more network requests while previously requested data is still in flight.
Another often-overlooked performance enhancement is jumbo frames. Jumbo frames allow higher bandwidths and reduce the required CPU use on the receiver side of the network. To be able to enable jumbo frames, both the sender and the receiver must support jumbo frames. The increased data payload from jumbo frames allows faster file transfers, reduces the traffic congestion on the network and improves the overall responsiveness of the network to the clients.
With the increasing interest in 10GigE network products, vendors are now working on new features to provide more performance and lower costs. One such feature, Remote Dynamic Memory Access (RDMA), will allow 10GigE fabrics to achieve even better latency performance, making it an even stronger challenger to the proprietary fabrics such as Myrinet and InfiniBand. This improved real-time performance will have benefits in the video edit space, where dropped frames are not tolerated. Improvements in the Internet small computer interface protocol will also drive innovation into the 10GigE fabric development.
As competition forces all industries to provide faster, more accurate products to be successful in the marketplace, the requirement for the power that 10GigE provides will meet that demand.
Corky Seeber is president of Small Tree Communications.