You are here: Home Page » Infrastructure » Infrastructure Archive » Video networks

Video networks

Dec 1, 2007 12:00 PM, By Glenn Hall

Move files in a tapeless workflow.

Tapeless workflows have arrived. But are your files in the right place? Do you need to get them elsewhere? Reliably? Quickly? The list of potential requirements is very long.

Developments in video networks now offer broadcasters an alternative to dispatching tape by courier. Signiant’s Media Delivery service controls intercontinental file transfers.
Click image to enlarge

We are so used to the reliability designed into technical broadcast systems, because, simply, they work, they're in real time, and they're reliable. Of course, that perceived reliability is the result of carefully established routes, relationships with telecom and satellite services suppliers, and multipath and failover design — all values deeply embedded into the very heart of current broadcast systems. Then file-based workflows came along, moving files via computers, from site to site, to and from remote sites, uploading and downloading — all usually done using some form of FTP.

Established and standardized through the Internet Engineering Task Force (IETF) and its predecessors, originally as RFC 114 in April 1971, the much-updated FTP today provides a simple transfer, with multiple clients, and many forms of added value. Such values are in security, authorization, efficient transfer, embedded encryption and notification.

On UNIX-like systems, a daemon runs and provides a service. There are thousands of variants, some tied to particular UNIX-like systems, some providing additional functions. Most, but not all, interoperate via the standardized protocol. Private versions abound, affording utility and even security through obscurity, though requiring root access to both ends for installation.

Today, there are many forms of IP-based transfer: FTP, managed FTP, transfer over HTTP, direct link technologies with security such as Point-to-Point Tunneling Protocol (PPTP), and more recently peer-to-peer (P2P) technologies, such as BitTorrent and Skype. (Skype has efficient file transfer built in and good firewall traversal.)

So what is happening in the file transfer market that is of interest to broadcasters? The answer is a great deal. Software technologies, services and DIY options exist, using cheap or open source forms of accelerated FTP. It's clear that there are several highly developed products and integrated services that lift the base level capability to the point of becoming a delivery system that achieves the levels of reliability required by broadcasters.

Myths about file delivery

First, let us nail down a few myths about file delivery systems:

Myth: FTP is not secure and loses packets. The reality: It all depends on what type of FTP you use.
Myth: User Datagram Protocol (UDP) is faster than FTP. The reality: Sometimes it is, but is has other issues.
Myth: FTP is used to load content to video servers. The reality: Those that are mostly used to load content to video servers are often private FTPs.
Myth: Video networks offer fast FTP services but don't integrate easily. The reality: That's a big myth; many providers offer APIs to their services today.
Myth: The software only works on Windows and is proprietary. The reality: Most vendors offer software on several platforms.

If you already have encryption, can you continue to use it? The answer is that transferred files are agnostic to their content; they're just bits. However, encryption in transit is a control on efficiency of transfer. Thus, reliable transfer with integrated manageability of encryption overhead is a high value.

File transfer acceleration

A need to back up large amounts of corporate data over various file links has emerged. Typically, a large organization, such as a networking equipment company Cisco, has a high value core of its own software that is in constant development, producing close to 1TB of daily changing data. Cisco's well-known router code is held and backed up on multiple sites, transferred frequently and securely by a Signiant system originally created by Nortel. The problem space addressed was simple to manage, allowing Cisco to reliably transfer its internal files across multiple sites, building in high efficiency, manageability and security, with tolerance of poor-quality networks.

To do this, Nortel turned to the team that had developed its own IP stack — the same stack that runs a large part of many international telecom networks today. This is serious, hard stuff and not the domain of small software companies. The requirements began in the corporate WAN backup area, attracted high investment and gained maturity. This is good news for broadcasters as these systems are now tuned for large file (video) delivery.

A needs analysis showed that files must be managed as they're transferred, and that a job ticket could be created that afforded manageability of the in-transfer session. Imagine initiating a terabyte transfer from home, or even at your station. It's likely you'd need some help!

Thus an important concept is introduced — initiating a transfer and then managing it all the way through to a final destination, which may be a video server.

Today in broadcasting, there are several companies offering services and technology that are built on these key values of reliability, manageability, security and acceleration, and in some cases with integrated workflows.

Technically, the majority of the acceleration component is achieved through a combination of repacketization, adjusting the delivery datagrams, taking into account the latency of the network connection in use and transferring the transport from the high overhead FTP to a lower overhead UDP.

Additionally, most of the software players have their own secret sauce, a patented code used in combining the compression algorithms with the packetization algorithms, thus providing a type of load shaping that moves faster. The math behind this is complex but is partly aided by the fact that cryptography and packetization both use similar mathematical approaches. In certain circumstances, they can be combined to provide an efficient transport.

Management is then the key, and this leads us to the nature of integration of such services, either through Web services over Simple Object Access Protocol (SOAP) or using a deeper API. This capability, which most providers offer to some extent, is the key to efficient transport integration, and to success in file-based transfer. In evaluating the integration of any such system, consider the integration of these services into your workflows. Deep APIs are attractive for complex integration. Most vendors offer a simplified product based on the core enterprise product with a straightforward start-up. Then they bring these systems into business use in short order.

Urgent delivery

Many of these software products originate from the complex world of telecom software and were developed originally for UNIX platforms. (UNIX began as a telephone operating system.) Today, companies offer fully compiled versions for the major UNIX-like systems — HP UX, IBM AIX and Sun Solaris — and for many Linux distributions, as well as a full compile for Windows servers. Core UNIX-like systems offer the advantage of efficiency right at the heart of the IP stack.

Some systems operate with node servers — which can act as an industrial P2P node, affording internode transfer as an additional function — with a management server, usually doubled-up for reliability. Thus a server is established at each major site and can send or receive, as well as initiate, local connectivity through a LAN. The management server communicates with the remote nodes and, by using network probes, adjusts the transfer as is required to be the most efficient, have the least cost or be the fastest.