Skip to content

Sure We’ll Test. It Won’t Make Any Difference, Of Course

January 16, 2020

Software development involves three important phases or roles.

Program manager: Designs the software

Developer: Writes the software

Tester: Tests the software

It’s interesting that Tester is the only one of those roles that’s name describes its role.

Working for Microsoft, the Program Manager set the schedule. Part of the responsibilty of a PM was to figure out how long it would take to create the software. The PM would also be responsible for deciding what features could be in the software. Every feature had an associated “cost” in time. The PM got to decide which ones made the cut.

Developers, of course, would make their best guess in telling their PM how long creating a particular feature would take to code. Every PM had their own formula for translating “dev estimates” into the schedule. I joked that the formula should be

Double the initial estimate and then move to the next unit of time

If a developer said something was only going to take an hour, figure it would take two days. I was right was more than I was wrong. Anyway, the development process was an ongoing conversation between the PM and the developer as features were added, or more often, cut from the software.

At some point the software was turned over to the testers. The schedule had an allocated amount of time for testing, of course. But, at Microsoft at least, once the testers had the product, the Test Manager owned the schedule. In other words, the product wouldn’t ship until the testing team announced they were done.

To Microsoft’s credit, they were relentless in empowering the testers. It didn’t matter if the entire world was expecting a product to ship at a particular date. If testing said it wasn’t ready, it didn’t ship. Of course, testing did everything they could to make the date. “Death March” is a phrase that often discribed this stage of the development process.

I don’t work in software development any more. But, the process is similar for other “project” based inititives. My current company is setting up a team of people to be our first line of support for outages.

Our current process for our first line of support outages is: Call Rodney. That’s been pointed out as a non-scalable, inefficient process. And Rodney’s wife is getting a little tired of his oncall status.

This new process isn’t anything radical. We just set up an 800 number that rings into a group that is staffed 24×7. Often there are two people on call, but during certain hours there is only one.

This shouldn’t be too different than our current process. After all, I’m just one person. So, how different could it be right?

Yeah, that’s what testing is for. We worked through the process prior to bringing the group online. We’ve been online for a couple of days, and it’s clear that testing prior to launch is not a 100% effective method for predicting what the launch will go like.

It’s not uncommon for me to be on two calls at once. One internal, one with the client. We discovered today that the way we’ve set up our new team only allows them to be on a single call at one time.

That’s okay, I’ll go ahead and run this one and be on both calls.

Occasionally, if we have multiple outages at the same time, I need to be on three calls at the same time. I use Skype for one on a headset that goes over my left ear. I use my cell phone for the second in a heaset that goes over my right ear, and I put the third one on speaker on my deskphone. All three phones have a mute button and so long as everyone doesn’t talk at once, I can make it work.

Yeah, that is way beyond what our new group is set up to handle. They didn’t even know it might required so they didn’t design the system with that in mind.

The advantage I have now is that I’m not running a software project that needs to ship to customers. We can “soft launch” our new support model with me still doing my role. But, I’m just reminded again that no amount of testing catches 100% of your use cases.

So, sure, you can test. Just realise it’s not going to make a difference.

Rodney M Bliss is an author, columnist and IT Consultant. His blog updates every weekday. He lives in Pleasant Grove, UT with his lovely wife, thirteen children and grandchildren.

Follow him on
Twitter (@rodneymbliss)
Facebook (
LinkedIn (
or email him at rbliss at msn dot com

(c) 2020 Rodney M Bliss, all rights reserved

Leave a Comment

Leave a Reply