I have been talking about this subject quite a lot recently with both customers and prospective customers. During these discussions there are a few points that have come to light by way of what’s important in the real world, and what peoples’ expectations are.
Firstly – None of the organisations I spoke with were only concerned about ensuring SAP was not adversely impacted by change, they all had a collection of applications and systems that work together. They of course need to be tested together at some point, yet very few set out to consider this at the start. While SAP and S/4 might be the most important to them, the organisations as a whole rely on a wider landscape which needs to be tested end to end as one.
This often involves different groups of people as well as different technologies, presenting both an organisational and a technical challenge. Customers explained how they used our management solution, Qualify, to organise tasks, tests and results across the groups and were able to pull this all together for analysis and decision-making purposes. Furthermore, where they have built test automation across multiple systems the validation of those connections was of immense value.
Secondly – Some companies had automation specialists and it is true that these people were the quickest to get going. But in some cases after a short while, productivity was greater in teams that were not technical but were strong in business and testing experience. They were able to use a code-free test automation solution and apply their knowledge after a short learning curve. So choosing the team with a long term view was important.
Finally – In most cases the main goal was regression testing which led to thinking about what that really means, and what value it brings. Amongst the prospective customers the focus was on how to determine what to test for, and of course this is incredibly difficult. It would be an impossible task to make assertions for everything that needs testing, and then if you did manage it, imagine the maintenance burden. However, among the clients using our test automation solution, TestDrive, this was simply not an issue. They had moved the goal posts by getting TestDrive to tell them about any meaningful difference from the last (live) release. Simply, “What’s different?”. That is very easy to achieve in TestDrive as it captures everything and can hence advise what’s new, different or missing. If you know this for a new release candidate, nothing will catch you out. It also coincidently makes maintenance very easy, in fact practically non-existent. If you want to know why, ask me.
George Wilson is SVP/Operations Director at Original Software