UAT - The Hidden Time Bomb

Image by Getty Images via @daylife

So many software developers are familiar with the four phases of software development. No, I don't mean the SDLC that spells out Inception, Elaboration, Construction and Transition.... More like the SDLC that starts with the "Honeymoon" phase, the "disillusionment", the "trenches" and "catching up to expectations."

We've all seen it... Everyone is genuinely excited at the beginning of a project.. Enthusiasm is high as requirements are written, the users seem engaged... and even though the requirements are a bit difficult to read or maybe slightly unclear, there is a feeling from the users that the team gets it, so we move along. Somewhere at about the 50% mark it settles in. The users become a little more involved and the team realizes that the requirements didn't quite cover all of the features that are going to be needed. There is tremendous intensity in this phase (the trenches). Finally, you think you are there. You're coming to the end of your budget, the schedule dictates that you should be wrapping it up and do final user acceptance tests (UAT). The software will be deployed in about a month and you are ready to deploy.

So, here we are. UAT. All of a sudden people you've never met show up at your door and politely introduce themselves. You smile, turn to your team and ask... "who is that, again?" - They don't know either. The first few tests work just fine. The first day was a success, but there are a few more days to get through. On the second day, the users come up with a series of tests your team is unaware of and magically, like a surreal nightmare, your development train runs over that penny on the rail, and the cars begin to fall off the track, toppling all over the place. Your project is WAY off the mark.... It doesn't do this, and it doesn't do that....

You have lost control. That wonderful project of yours that you've reported as "Green" in your status reports for the past 6 months is now "RED" ... Blood red. You are caught off guard... you have lost your balance... you are in a free fall and you can't even see the floor. Sure, if we were in outer space there would be complete silence, but we're not... so management hears you screaming "Noooooooo!" as you fall, down... down... down... no end in sight... just a faint echo as they look into your development well and you fade from view. (dramatic, huh?).

Well, unfortunately, this is not all that uncommon. This happens a lot. UAT is the gotcha for the development team. There is nothing more in this world that raises the hair on my neck than a UAT surprise. As a project manager or development lead, UAT is the validation that you got it right, and for some reason, everyone gets so preoccupied with the development of the project that they miss the most important part of any software development... the acceptance of the code.

I can tell you, I have seen the best projects fail in UAT. Perfectly delivered, meticulous projects, that when reviewed were absolutely delivered to the contract. The development team did it, but the client wouldn't accept it. The worst case that I can remember ultimately took a small consulting practice out of business as UAT went on for months beyond the fixed price contract. The practice prevailed in arbitration, but the costs were overwhelming and the office was shut down. Perfect software development proven in an arbitration resulted in a failed consulting practice. Unfair. Oh, and in addition, the client who decided to go to arbitration lost a lot of money as well chasing the consulting company.

So, how can one avoid these kind of problems? Even if you deliver your project perfectly, as my friends in Dallas did. The first thing you need to come to terms with is that your contract should go well beyond the development of the project. Its easy to say, "we did everything right," they just don't want it, or its not my problem they don't get it. I guarantee that at a minimum you will pay, your reputation will suffer, your business will suffer.

The UAT phase is the most critical phase of your project. Does that sound ridiculous? Think about it. If you can't control the UAT, you are entering the "financial loss" phase of the project. And, let me be clear, when I say control, I don't mean "win the test of wills." I mean understand the amount of time required, the potential to make critical fixes, and assure the software will be used and accepted by going through ALL of the pertinent test cases required to accept it.

The biggest flaw I see in software development is the poor preparation teams make towards UAT. The best way to avoid UAT time bombs is to get your users to pony up and agree to all of the test cases they will run. Immediately challenge them when they identify them. I've listed questions below that you need to find answers to, and formally agree to, immediately after requirements and no later than the beginning of development (months ahead of UAT). Additionally, you need to revisit this list multiple times throughout development to make sure there will be no surprises in UAT. If you find that the answers are changing or growing, you need to plan and budget accordingly:

1. Who will be performing the UAT?
2. Do those identified represent 100% of the users?
3. What specific cases will you run? (get a list and full understanding of conditions that will be checked)
4. Are we certain that those cases will cover 100% of the acceptance ?
5. Which exception cases do we need to run in UAT? (outside of normal flow)
6. Are there any more?
7. When I document these, will you be able to provide an authoritive signature that states that successful completion of these tests constitutes acceptance?
8. If not, then lets keep working this document until you can.
9. Who will provide the final signature on acceptance?

The development of a well written and understood UAT plan is validation for your requirements. If the UAT goes beyond your requirements documents, your team missed the objective, and the software will not be accepted.

Furthermore, assign a lead who is skilled in change management to be the responsible person on the dev team for a successful UAT. This person should be meeting regularly with the client's organization to show them work in progress (what the system looks like and how it will be used) months in advance of the software's completion. They should understand concerns and help promote the use of the software. The feedback they receive should be shared with the development team and taken very seriously. When replacing an existing system, they should also take the time to understand the older system so that important features the users want/need aren't overlooked.

What UAT surprises have you experienced in your career? How could those have been mitigated? I'd like to know.