Plan with a STEADY hand
Oct 1, 2006 12:00 PM, BY MICHAEL WRIGHT AND BRIAN REDMOND
Even a small change to one piece of equipment could ignite a chain reaction down the system line.
Transition.
To a broadcaster that word can mean anything — activating a new cell phone, moving from one building to another or a major system upgrade.
Perhaps you don't worry about transitions. Maybe despite never making detailed plans for changes to your operation, you have been lucky enough to avoid disaster. Even so, chances are that sooner or later, you'll encounter a system change with the potential to create substantial problems.
This article explores some of the situations broadcasters could encounter, particularly in dealing with today's server-based, IT-centric systems. A seemingly simple software upgrade or transition can turn into a major undertaking fraught with potential disasters. More importantly, you'll get some practical advice on how to pave the way to a smooth upgrade from two people who've been there.
No problem?
“No problem,” says the equipment manufacturer. “Everything will go just fine. The software has been fully tested and is working perfectly in other locations.”
Even for supposedly simple transitions, whenever you hear, “No problem,” “It's just a simple upgrade,” “It won't take long at all,” or “Don't worry,” beware. It's time to take a good look at what you're about to undertake and plan the change carefully. Don't think that such problems only happen to other people.
Mitigate risks
Let's take a look at some of the risks in a system upgrade and what to do to mitigate as many of them as possible by following a careful transition process. Take a look at the example system in Figure 1 on page 80. The system uses two network switches, one for the broadcast high-res system and the other for the newsroom systems. It is a good practice to keep the broadcast system separate from the office systems.
For now, let's assume that this system has been operational for eight months. During that time, there have been several known bugs and issues, some of which are fairly critical and others of a lower priority. Finally, a new software release that claims to fix many of the bugs and, specifically, some of the critical issues the operations people have been putting up with and working around. It's time to plan a clean, trouble-free transition.
Read the instructions!
First read the release notes carefully. Compare the issues the new software fixes with the system's critical issues to ascertain which of these the new release addresses. Next, determine which of the other, less critical issues in your system the new release is supposed to fix. Then, ask these key questions:
-
Does the new software add or change any of the system's features or functionality?
-
If there is a change, will it affect the workflow?
If your answer to either of these is “yes,” you will need to communicate with the experts whose areas these changes impact. For instance:
Devilish details
-
If there's a potential impact on the system's high-res editors, you will want to discuss the change with the lead editor for this area.
-
If a change affects the metadata server and it affects archiving or the process by which you have been purging or searching for files, you'll want to talk with the operations lead for that function.
-
If the new software affects the database, you'll need a thorough analysis. A change that affects the database can also affect search procedures. That, in turn, can have an effect on any device that has access to the media and on how each device interfaces with the search function and uses data.
To keep our example simple, let's assume that there are no feature changes and no changes to the overall system workflow. This new release simply fixes bugs.
First things first
That makes everything pretty easy, doesn't it? Maybe. But take a closer look, and you may find additional hurdles.
Because the edit interface is affected in our example, it requires an upgrade of the main file system controllers in the SAN servers before you can upgrade the high-res editors. In addition, upgrades to the ingest and playback servers and the database server cannot be undertaken until the SAN server has been upgraded. Furthermore, another change — this one to the database server software — must be completed before the system can handle I/Os and high- and low-res editors.
Still another issue to address: This upgrade affects both software and hardware. For example, you'll need to increase the RAM in the SAN servers to support the software upgrade.
What at first appeared to be a simple upgrade, and one that may well be quite logical and fairly simple to actually implement, has turned out to be more complicated. It will be time-intensive and must be carried out in a precise order, or it won't fix the problems it is meant to address. In fact, doing things in the wrong order could introduce more issues.
| Want to use this article? Click here for options! |  |
