There have been a few occasions now when I have had the pleasure of hosting a round table discussion on User Acceptance Testing (UAT). I find it interesting because to many in the world of Software Quality and Testing, it is somehow not “proper testing” or not really part of the important task of delivering software changes. You don’t see pseudo academic papers on it with interesting formulae to calculate metrics such as the mean time between failure or percentage of code coverage. The whole subject is just not meaty enough for the hardened professionals. And perhaps there is some truth in that. The last time I was involved in such a discussion one of the standout tips for success was “cookies” or “biscuits” as we say in the UK. I can’t take any credit for this insight, but I was happy enough to borrow the idea for the most recent event.
The round table last week was held at a user conference for Infor M3, LN and Baan users. If you think about that situation, most of the “meaty” testing will have already been done further down-stream in development, system testing, integration testing and perhaps even regression testing by Infor, or whichever vendor’s products we are talking about (other ERPs are available…). They have probably used continuous build, continuous integration and continuous testing as well as other automation to maximize the chance of defect detection.
So, could you argue that any more testing is a waste of time? Maybe. But that might be valid if the vendor was going to underwrite or compensate you for any errors that did get through and had a negative business impact. But they don’t of course and the buck stops with the users of the application, so we understand the desire, the need to make sure it is right.
Then if you consider,let’s say, a significant change such as an upgrade, how many users might be involved across the business in testing that, probably over a number of weeks and cycles? 20? 50? More? And that is per company, so if there are 1000 companies using that particular solution does it turn out that most of the testing taking place in the world is UAT? Is it the biggest form of software testing there is?
In that context, these were some of the points that were raised in the discussion on the subject.
UAT is a substantial amount of effort and cost, but you still need to do it. Users don’t like to do it; IT teams can’t do it like the users and don’t want to do it either. So where is the compromise?
The ideal position is to have an easily created and easily maintainable automated regression test suite that is run by IT on behalf of the users. It finds all the differences, wherever they are, even where you didn’t know to look. This tells you what you need to look at, what in some cases the users will need to look at and determine whether there is an issue or not. It might mean just a revision to a standard operating procedure or training content. When the users are testing, they are empowered with tools that make this task easier for them.
If you are looking for a simple, easy to implement solution to your UAT challenges you might like to watch this two minute explainer video or request a free trial. Would automated regression testing make a difference in your organisation? Find out more here.