In the 1972 movie The Candidate, Robert Redford plays an idealistic young man who is recruited to run against an incumbent senator. Through the shrewd and calculated efforts of his campaign team, the young neophyte emerges victorious and closes the movie by asking his team, “What do we do now?” Other than, maybe, stirring you to make a mental note to add this to your list of movies to watch on Netflix, you’re probably wondering how this relates to the principles of a data center operations plan.
The point of intersection between a 45-year-old movie and a data center operations plan is just this: in both instances, the vast majority of planning and effort are on the design and development of the structure — a campaign in the former and a data center in the latter — with little focus on what happens after their completion. Data center operations often appear to be the ad hoc marriage between personnel, a few rather large three-ring binders, and in the case of MTDC’s a service-level agreement (SLA) presiding over the whole affair. Improving your operational planning, or your providers, requires a focus on five fundamental principles to ensure that the first question you ask after taking the keys to your new data center isn’t, “What now?”
Principle 1: Experience is the best teacher. Like so many of the things in life, looking in the mirror and admitting that you’ve made a few operational mistakes in your career is the first step to making these experiences lay the groundwork for understanding what you should, or shouldn’t do, in operating your data center.
Principle 2: Designed through the eyes of an operator. If you’ve greeted this one with a “Duh,” I encourage you to look at your shelf of three-ring binders and try to remember the last time you looked at one? The Japanese have a term, Poka Yoke, that best describes the ultimate goal in the development of processes and procedures. Roughly translated it requires focusing on the lowest common denominator to reduce human error. In other words, make it idiot proof. Let’s face it, when over 70% of outages are traceable to operator error we all still have a long way to go on the whole simplification thing.
Principle 3: Flexibility and control. While seemingly broad, this concept is quite straightforward. The scheduling of activities by your, and especially, provider personnel must align with your operational tempo. Operational requirements must revolve around your specific needs and not the other way around.
Principle 4: Training and certification program. The method for achieving the goal of organically generating a more confident and competent maintenance staff requires a role-based training program comprised of:
- A formal curriculum
- An objective measure of understanding
- On-going processes that are continually updated and refined
Principle 5: Focus on eliminating errors. Requiring a technician to attempt to diagnose and repair a reported CRAC problem while holding a flashlight in one hand and a three-inch manual in the other is not conducive to quick and effective resolution, and yet this typifies the standard mode of operation in many existing data centers. Obviously, in this situation, the opportunities for the introduction of human error are myriad.
One mode of addressing these issues is providing “hands-on” personnel with greater access to information in a format designed with the specific intent of minimizing or eliminating the potential for human error. For example, working with a vendor that specializes in converting all your procedures into step-by-step digital checklists that each technician can access from an electronic device.
Planning for the operation of your data center is a critical, and often, overlooked element of the new data center process. Proper operational processes and procedures don’t result from rigid adherence to past modes of operation or ad hoc efforts devoid of any underlying guiding foundation. Developing an effective and adaptable plan for successful operations requires the adoption of specific principles to guide these efforts, and IT needs to think holistically about their operational goals and how they intend to achieve them.
