When professional media systems are delivered, they frequently contain computers and servers, and before we know it, because we installed them and we maintain them, we are the go-to guys and gals for all things computer-related. There are several problems with having computer system administration sneak up on you:
Bad things can happen to good broadcast engineers when they start heading down this road. On the other hand, it is inevitable that highly skilled broadcast engineers will end up performing system administration tasks. Let’s dig a little deeper into the issues raised above.
You probably are not trained to do the job. As a professional media engineer, there is no doubt that you have worked quite a bit with computers. You may have even taken some courses on computer architecture, programming and so on when you were at school.
But being a system administrator is a different animal. You are not just using computers; you become responsible for ensuring that others can use them as well. You will have to understand server operating systems (OS) at a whole new level. In some ways, a server OS is similar to a desktop OS. On the other hand, there are special utilities and tools that help you control access to the resources on the server, monitor security and configure the server for different applications.
As a system administrator, you may need to delve into the world of *NIX (Linux, Unix, Ubuntu, BSD, Red Hat, etc). There are several challenges with *NIX administration. First, the most efficient way to manage these servers is through the command line. This means learning a whole vocabulary of commands such as SED, LN, GREP and so on, many of which do not have equivalents in a Windows or MAC world. Second, many of the versions of *NIX are just slightly different from each other. So, you may learn where system files are kept in a Linux system, but when you move to BSD, things are in a different place. These are just a few points where a lack of training can hurt you.
Although professional training as a system administrator may not take care of all these issues, training is a wonderful thing, and it should be a planned, budgeted and expected part of the job.
The job may involve critical on-air or production systems. When you combine this with the fact that you probably aren’t trained for the job, you are headed for trouble. After all, if you are not trained, and then you are asked to maintain critical systems, how are you going to learn? I have frequently heard it said that the way someone becomes accomplished at anything is to have worked at it long enough to have made a number of mistakes, and to have learned from them. If you are maintaining critical on-air systems, how many mistakes can you make before your manager removes you from the opportunity to learn and go back for more experience? Not many; I can tell you from personal experience. And a related question: How can you make changes to a critical system without having confirmed those changes in a test environment ahead of time? Frequently, when we are thrust into a system administration role, we do not have time to think these issues through.
You may be unaware of architectural patterns and anti-patterns. Again, this relates back to the first point. If all you have is on-the-job training, how can a manager expect that you will be following best practices for system administration? After all, these best practices evolved through study and analysis of system administration tasks over years. Without training, even the sharpest knife in the drawer is going to make mistakes, simply because no one can think of everything.
Architectural patterns are ways of doing things that have proven to work well in most cases. An example of this is two-factor authentication (e.g. something you have, and something you know). Many different approaches have been tried to authenticate a user, and two-factor authentication is a successful pattern. Think of your ATM card; it involves something you have (the ATM card) and something you know (the PIN). There are many successful architectural patterns for system administrators, and books have been written on best practices.
As you might expect, anti-patterns are designs that do not work well. They are designs where you might say, “Given the opportunity, I would never do that again!” An example of an anti-pattern, although you see it all the time, is a watch folder. Watch folders — folders that are watched by an application such as a transcoder — are frequently used as a way to tell a downstream device to do something with a piece of content. For example, drop content into a folder, and magically, a transcoder will pick up the content, transcode it to another format and drop it in a pre-determined location. What’s not to love? They are easy to configure, easy to understand and easy to change.
The big problem with watch folders is that you are using a file system as a messaging system. I could write an entire article on this topic (and maybe sometime soon I will), but this is a bad practice. If you want to tell a transcoder to do something, wouldn’t it be better just to send it a message? Then it could acknowledge the message, and you would have an audit trail. What if someone puts a file in a watch folder, and then deletes it after the transcoder has started the job? How does the transcoder know you are done writing the file into the watch folder?
Getting back to issue number one, training may help you learn about patterns and anti-patterns without incurring the scars of practical experience.
Management may not be aware you are serving in this capacity. This is a problem because you are essentially performing two jobs for the price of one. Not only that, but when you make requests related to system administration, your manager may have a difficult time understanding what is behind your request. This might not only hurt you, but it may hurt the manager and the company (the law of unintended consequences). It is much better if everyone, especially your manager, understands that the business has changed, and that system administration is now a formal part of your job description. If you have found that system administration has crept into your job over the past several years, be sure your manager is aware of this.
As stated earlier, there is probably already someone in your organization who is the “real” system administrator; he or she is likely located in the IT department. This can cause no end of problems. Again, communication is key. I have seen many a good broadcast engineer get caught up in the turf war between IT and broadcast technology. If the organization is not clear about roles, then top management may question why in the world two IT departments are needed. As a broadcast engineer, it may fall to you to help educate your manager about the role you have found yourself in and how that differs from regular office IT.
Lastly, if you find yourself in the role of system administrator, please remember you are there to serve the needs of the organization; don’t become a pain. Sometimes people who are in charge of systems treat these systems as their own personal kingdom. You will do yourself and your career a huge favor if you keep in mind that, without the creative and business types who frequently use the systems we administer, we would be out looking for work.
—Brad Gilmer is president of Gilmer & Associates, executive director of the Advanced Media Workflow Association and executive director of the Video Services Forum.