That Test Did Not Test What You Think It Tested

The product launch was going well. We started taking calls right on time. A rarity. Our software was working flawlessly. But, several agents weren’t logged in. 

Clark, what’s the deal with those agents in the second row?

Oh, they can’t get logged in.

This was bad for two reasons. First, as the Technical Account Manager, making sure people could get logged in is kind of one of my responsibilities. Second, this was first I heard about a problem. The day of the launch is not the day to being finding these type of issues. We not only have processes designed to make sure everyone’s account is properly configured prior to launch, we even have a final check the day before the launch.

Did you have them test their logins yesterday?

Yes.

So, what happened between yesterday and today?

Nothing. They were broken yesterday too.

The answer caught me off guard. It’s true they had tested the logins just like we had designed. I didn’t think to include a step in that testing describing what to do with a failed login. 

We quickly collected the list of impacted agents and sent it off to our client contacts to expedite. We also figured out why Clark had an issue. He knew the accounts were broken. They had been broken for the entire training period. He had been trying to get them fixed since his class had started. What went wrong? 

Having a well defined process.

We run countless training classes. Our trainers, in addition to teaching the class, make sure that the names of the agents are submitted so that the agents have access to the proper tools. We literally have dozens, if not hundreds in any given week. Most times the request goes through just fine. But, this was the exceptional case.

What do you do when the normal process fails, or is delayed, not just in this instance, but in general? Many people stand at the door and keep knocking, hoping that it will eventually open. Some people head around to the backdoor to see if that one has better luck. Still others make their own door. 

I’ve always soft of been one of those last types. It’s why I enjoy my job. My role is to find ways to resolve problems. It’s expected that I will then design processes to avoid the problems in the future. But, much of my day is in troubleshooting mode. 

Unfortunately, our trainer didn’t know that. And I’m not the only escalation point for issues. We have multiple people solely, or partially tasked with getting the door open. Our trainers didn’t use any of us. 

It worked out okay, our launch went well, we quickly got the list of agents to our client and they expedited the account creation. But, it was a good reminder that the best contingency plan in the world is worthless if people don’t know to use it. Simply testing the system is only effective if you have a process for both a successful and an unsuccessful test. Otherwise, you’re just having people go through the motions without any real benefit. 

Rodney M Bliss is an author, columnist and IT Consultant. His blog updates every weekday at 7:00 AM Mountain Time. He lives in Pleasant Grove, UT with his lovely wife, thirteen children and grandchildren. 

Follow him on
Twitter (@rodneymbliss) 
Facebook (www.facebook.com/rbliss) 
LinkedIn (www.LinkedIn.com/in/rbliss)
or email him at rbliss at msn dot com

(c) 2016 Rodney M Bliss, all rights reserved