Skip to content

Datacenter CSI: The Day The Servers Died

December 23, 2013

Best of 2013. This was the 6th most popular blog entry of 2013. Originally posted on May 24th.

Rodney M Bliss

You know that little inkling you get that something somewhere isn’t right? It’s a skill that really good datacenter managers develop. Almost Jedi-like.

I feel a great disturbance in the servers.

Well, today was not one of those days. Today was more like when you know your house is on fire because of all the smoke, flames, firetrucks and especially the alarms going off.

“We just lost another one. We’re at 69% of power supplies on the Cisco appliances.”

“That’s good right? We’re still above 50% so, we haven’t lost any servers yet, right?”

“Well, it WOULD be good if the power supplies would fail uniformly. But, we’ve lost both the primary and the secondary in several chassis.”

“Do we even know how much is down?”

“We’re working on it. . .There went two more. We’re at 63%.”

This was a full-on meltdown. And the frustrating part was that we…

View original post 929 more words

From → Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: