Skip to content

Thank You For Breaking Our Systems?

January 22, 2015

Great job on solving today’s problem, Rodney.

Yeah, Rodney you really saved us on that one.

You’d think I’d be grateful. My boss and my coworkers were telling me what a good job I did resolving a nasty technical issue. I’d just spent two days working with my engineering teams to do an emergency install of a redundant backup system that had failed when we tried to use it the day before.

But, I wasn’t feeling particularly worthy of their gratitude. I was the technical project manager. The word “technical” is right there in the job title. I had run this project. I had vouched for the validity of the technical solution. The very same technical solution that had failed to work the day before.

Every project is a series of tradeoffs. You never have all the resources and all the time and all the features you want. You have to trade off. It’s called “Risk Analysis.” And during the project we identified the risks around our backup solution. The project team considered the issues, weighed the risks and we made our choices.

And when things started to go south the day of the rollout, the risks we had evaluated as low turned into high risks. So, I was looking at the problem and recognizing the choices we could have made months earlier that would have prevented our current outage.

Was I responsible?


There were also technical issues that had led to the outage, but I couldn’t get past the decisions we had made months earlier. The risks that we had noted and dismissed. I didn’t feel like they should be thanking me. If they had accused me I might have felt they were more justified.

I wanted to have properly prioritized the risks.

As I was leaving after a 12 hour day with the last of the technical issues finally resolved, the site manager thanked me again.

Great job today Rodney.

Thanks, Derek. But as the technical PM, I figure I’m the one who should have prevented it in the first place. Not feeling really brilliant. . .or competent.

You don’t get it, Rodney. Stuff goes wrong. It’s why we have jobs. But, look at the last two days. When stuff fell apart, you didn’t. You kept your head. You put together a contingency plan. You put in long hours and you led the engineering teams in building a solution. At the same time you managed the client and kept our management team informed. You aren’t being thanked for causing the problem. You are being thanked for the professional manner in which you resolved the problem. That’s valuable. And people noticed. Nice job.

Thanks, Derek.

I really hadn’t thought of it that way. Everyone makes mistakes. How you handle yourself when the system falls apart defines you.

You are at your best when things are at their worst.

Rodney M Bliss is an author, columnist and IT Consultant. His blog updates every weekday at 7:00 AM Mountain Time. He lives in Pleasant Grove, UT with his lovely wife, thirteen children and one grandchild.

Follow him on
Twitter (@rodneymbliss)
Facebook (
LinkedIn (
or email him at rbliss at msn dot com

Leave a Comment

Leave a Reply