Spring rains bring on more than just flowers or in my case, weeds.  The phone started ringing early the other morning.  My coffee was still brewing when the continuous ring of the phone demanded me instead of the regular answering service.

It would seem that lightning hit a pole close to one of my clients.

Lightning is far from respectful of your deadlines or the amount of work that your staff has lined up to accomplish.  From simple power outages to fire, lightning all by itself is a disaster in the making.  Some simple steps ahead of time can keep your company from being a victim to what this client was.

One girl had her headset in when the lightning struck and was shocked. Happily, she is ok, but their systems were not so fortunate.  Had the grounding been worse; she may have been the path to ground.

Once the power was restored the server, router, and switch, did not recover.

The one machine on a UPS died as the power went out.

What went wrong?

Surge protectors have a finite lifetime.  People buy these power strips with surge protectors and forget about them.  Surge protectors are nothing more than a power strip with something in them known as a “Metal Oxide Varistor or MOV.”

Any power surge above an acceptable voltage is clamped or shorted to ground by this device.  The problem is the MOV only last so long before it no longer functions.  Every time there is any spike in the line from compressors shutting off to other electronic “noise” these components are adversely affected.

What is better?”

A UPS of enough wattage to allow the computer to be safely powered down in the event of a power failure.  Along with the backup power ability, these devices have more sophisticated line conditioning circuitry protecting your equipment from stray voltage spikes.

One note to remember, these too only last so long before they must at least be maintained, or replaced.  Any CIO worth his salt is familiar with Hardware asset management and has this is mind for his budget.  CEO’s hate surprises like unexpected expenses.  It is much easier to argue a budgeted expense than going hat in hand begging forgiveness for your ineptitude.

Suffer a catastrophe like this client, hope your boss does not hire someone like me to do a root cause analysis.

At the very least batteries must be changed out but keep in mind that an MOV is also part of that piece of hardware.  I would budget the replacement of a UPS, rather than just the batteries if it were me.

Unless you have electrical engineers on staff, who are qualified to re-certify that equipment, it is too cheap not just to replace it.

 

Along with outdated hardware or not enough of it, I have seen too many times the ground plug defeated to save a dollar from an electrician.   Those ground plugs are there for your protection, not because someone wanted to make it difficult for you.  The problem with temporary is all too often it becomes permanent.

Lightning struck outside one of my client’s offices hitting a pine tree.  Finding the electrical ground for the building, which was poorly grounded, everything in the building suffered a power surge knocking out much of their equipment.

Many times, building management will only do what is necessary by code and leave the gamble up to you the tenant.

Depending upon your location, achieving a good ground could be difficult.  The type of soil must is taken into account among other things. Again, depending upon your location, you might want to invest in grounding your building with lightning protection equipment including lightning rods or now they call them “air terminals.”  The idea is to have some amount of confidence that if lightning hits, it will strike your planned target and be dissipated safely into the earth.

Since all computer equipment and now phones are wired through the network, this last customer lost computers and phones along with the network infrastructure.

Failure to plan is planning to fail.

The cost of the hardware and time to repair was minimal, compared to the amount of time the company was out of business.

Insurance will only get you so far.  As these spring storms fire up, there is a real element of danger to your building, business and, like the one young lady found out, to her person.  Had proper grounding been utilized I doubt the girl would have felt the shock in her ears.

While a tested, reliable disaster recovery plan will allow you to sleep at night, preventing the disaster in the first place is what you should shoot for.  That starts with planning.

From your building security to network security, right down to protecting your infrastructure from mother nature, accounting for every contingency is paramount.

Truth told, there are seldom good surprises in business.  Mitigating the surprises with proper planning can prevent poor performance.  Asking “what if” is key to any plan.  Weighing cost vs. probability allows anyone with some business acumen to make sound decisions without breaking the bank. Understanding the risks, are the starting point.

 

-Best