Fire, Flood and Famine!

I’m a man of faith, but even if I weren’t I think I still would have said a little prayer before clicking the START button on my migration servers. It was 5:00 Friday afternoon. The start of “Migration Weekend” at MeritCare hospitals. For the past eight months we’d been planning the migration from Microsoft Exchange 5.5 to Novell GroupWise. All the planning, the meetings, the trips to Fargo, ND (You Don’t Get To Pick Anymore.) All of it for this moment.

I’d run my tests.

I’d built in my buffer.

But when it came right down to it, there were no guarantees that our plan was going to work. No one had ever tried a migration this big, and we had to complete it in the next 36 hours.

If it didn’t work, there were going to be a whole lot of angry people who’d spent a whole lot of money. My reputation. . .Novell’s credibility. . .

I was stalling and I knew it.

Okay. Here goes.

CLICK

Not sure what I was expecting, the GWMigrate tool on Workstation1 grabbed the first batch of 50 names and started converting them and moving them into our brand new GroupWise post office. I quickly started the other 9 Workstation machines and got ready to deal with the first error messages. (There are always errors that crop up.)

Tim, if you want to go kick off the machines in the lab, I think I can cover these.

I knew the first 30 minutes was crucial. If we made it past that point, it would indicate that we’d worked out the show stopping bugs.

I sat transfixed by the animated icon that was moving “Microsoft blue” messages though the converter to “Novell red” messages.

After about 10 minutes Workstation3 suddenly flashed an error. It had found a corrupt account and was announcing it was skipping that account and moving on. I noted it in our log and started to breath a little easier.

Maybe we’ll make it through without. . .

FIRE

BEEEEP. . . .BEEEEP. . . .BEEEEP

In a panic I scanned my servers. They were all happily humming along chugging through the first 500 of the nearly 7,000 accounts we were migrating.

BEEEEP. . . .BEEEEP. . . .BEEEEP

That’s not a computer sound. What is that? It’s really loud!

BEEEEP. . . .BEEEEP. . . .BEEEEP

Fire alarm. The fire alarm is going off and I’m tucked away in the basement of a former hospital that had been converted to offices.

I looked at my servers. I knew that the minute I walked out the door they were going to start throwing errors. And even if they didn’t, we’d broken up the user list and were feeding them to the migration servers in 50 user chunks. All of my servers would need a new file within the next 30 minutes or so.

No job is worth dying for. . .well, no computer job. . .but, I’d put too much time into this project to run at the first sign of trouble.

(Yes, I was actually talking myself into staying in a burning building. Crazy? Stupid? I know. But, it was pretty important.)

Finally, with a last glance at the servers I reluctantly opened the door to the hallway. . .and immediately was hit by the smell of burning popcorn.

No way.

No way am I leaving for burnt popcorn. Besides Workstation6 had just flagged another problem account. Trying to block out the noise, I went back to babysitting my migration servers. On our webcam in the lab, I saw that my coworker Tim had the same idea. He was hunched over a server reading through an error report.

Eventually the fire alarm stopped. I really can’t tell you when. I presume at that point they told everyone who had actually evacuated that it was safe to reenter the building. I heard the firemen were pretty upset to get called out for burnt popcorn. I knew exactly how they felt.

FLOOD

We survived the first half hour and Tim and I settled into a routine. After about 5 hours we hadn’t seen anything that we hadn’t seen in the testing. The smell of burnt popcorn was nearly gone and we started to relax a little.

Hey, Rodney is that normal?

Huh? What?

That water soaking the carpet under the migration computers.

What?

Sure enough, a flood was slowly spreading from one corner of the room and getting dangerously close to our migration servers. One of the MeritCare employees on the project spoke up.

Yeah, that happens every time we get a really heavy rainstorm. It’s why we never put offices in this space.

Well, we can’t let those servers and power cords get soaked. But, we can’t disconnect them either. They’re right in the middle of a cycle.

The solution was we put the power strips on the tables and we jacked up the servers on blocks. Pretty soon 8 or our migration computers had acquired wooden stands.

Will it flood the whole room?

Nah. It’ll just soak that corner and about 10 feet around it.

Got it.

Pretty sure we didn’t cover this in the project meetings.

FAMINE

Okay, we didn’t actually have a famine unless you count the loss of the popcorn. Like most computer projects, there were generous amounts of pizza, chips and caffeine in various liquid forms.

It wasn’t until much later that I remembered to ask our project manager about the fire alarm.

So, what happened?

You guys remember that you asked me to keep Henry, from corporate out of your way, right?

Yeah, he was never part of the project team and if we even let him in the room, he’d insist on helping and then we’d have to fix whatever he screwed up.

Well, I told him that. . .but nicer. Anyway, he figured if he couldn’t help, he’d make everyone some microwave popcorn. The package said, “Five minutes on high.” So, he set it for five minutes and then walked away. He’s not real familiar with microwave popcorn.

Next time maybe we should find him something to do.

When we finished Saturday evening, we realized a migration tool had an undocumented bug that forced us to do the entire migration a second time. . .without the fire and flood.

This week I wanted to explore some of the stories around consulting. I explained that consulting is like Feasting on Brownies. . .Every Three Weeks. Then, I related how I went about Setting Consulting Rates. I told the story of how I once Billed 25 Hours In a Day and how I fixed it. Finally, I described a time our client paid $500 Dollars An Hour for us to do nothing.

About the Author
Rodney M. Bliss is a blogger, author and IT consultant. He has now made it a goal to exit buildings when they might be on fire. He lives in Pleasant Grove, Utah with his lovely wife and 13 kids. Follow him on Twitter, Facebook, LinkedIn or send him email (rbliss at msn dot com.)