The Invisible Administrator

When I was first learning system administration, a senior admin told me “Good administrators are invisible”. I did not really understand that at first, but now it’s one of my prime tenets. For the record, being invisible is very hard. The projector operator is invisible, until the movie cuts out. Likewise the train conductor, the switchboard operator, and the air traffic controller. Invisibility hinges on a few talents, the biggest of which I will attempt to outline.

The first part of being invisible is planning. Always have a backout plan, and always plan your disruptions around times of low usage. These things require a mixture of common sense and data analysis. Monday mornings, plan on more email to flow. Likewise Friday afternoons. Unless, of course, Friday is, say, Christmas, in which case plan for the rush on Thursday. Data collection and parsing is your friend here. My personal favorite is Munin, which I plan to write a better article about soon.

The second part is anticipation. Anticipation is largely tied in to planning, but is much more organic. You have to have a holistic, insightful view of your customers and know what they will want before they do. When people bring you solutions to implement, make sure you understand the problem they are trying to solve. Often you will find your skills and experiences can save a lot of pain and offer a more elegant solution.

Lastly comes rhythm. You have to know the rhythm of your machines. You can bet a truck driver can feel when his rig needs an oil change.  Be that guy and the oh-noes 2am-on-a-Saturday problem can be a ho-hum sometime-after-lunch-on-a-Thursday task. Knowing the heartbeat of the datacenter can help you jump to problems immediately. Human brains are excellent at recognizing patterns and extrapolating probable results.  It’s one of the reasons the machines aren’t adminning themselves. Use this capability and try to keep abreast of everything that occurs with your machines. Dropping packets? Maybe Tommy moved an ethernet cable to close to a flurescent light when he was doing that cable pull two days ago. Without these little tidbits of information, you can’t see a pattern. 1,2,?,?,?,? is not as useful as 1,2,4,?,?,32.

Hopefully you’ve found this useful, or at least thought provoking.

July 29, 2009 • Tags: , , • Posted in: Blogging

Leave a Reply