Case study: CertainTeed

A Revolutionary Approach to Regression Testing

Zero defects in production

Marc Croquette, an IT Director at CertainTeed is responsible for multiple, heavily customized SAP implementations across 5 divisions. Read how his team ensures that every change is entirely fit for purpose

A high rate of change overwhelming the users

In addition to SAP there are numerous integrated applications including Salesforce, Manhattan Associates and Hybris. Changes come thick and fast from multiple sources while overlapping projects from different business units compete for scarce analysis and testing resources.

Business users were being asked to abandon their day jobs and repeatedly assist in the verification of the changes. Some teams were scattered across a range of time zones exacerbating communication and collaboration.

The solution would need to check every item on every screen and report all differences

Marc knew they needed to test the areas that had changed, but as importantly, they needed to test that defects had not been unwittingly introduced as a result.Marc likened the process to an airport luggage carousel, where every difference found in a regression test is automatically captured and tracked in the same way that all the baggage from an aircraft arrives on the carousel. Business analysts (just like passengers) would ‘collect’ the differences they expected and anything still on the carousel would be an unexpected change needing investigation

CertainTeed is a North American manufacturer of sustainable exterior and interior building products.  It is a wholly owned subsidiary of Saint-Gobain, the world’s largest building products company. www.certainteed.com

The Challenge

Users were overwhelmed by a high rate of change. Necessary checks didn’t always happen, were very costly and inefficient to run, leaving room for defects to reach production

The Baggage Carousel

In the same way that all the baggage from aircraft arrives on a carousel, Marc envisaged every difference found in a regression test being automatically reported the same way

Our Solutions

  • TestDrive

  • TestDrive-UAT

  • Qualify

The Solution: Original Software’s quality suite

TestDrive delivered two vital capabilities. First it captures every item on every screen automatically, without any coding. Secondly, TestDrive allows the results of a regression test to be compared to any selected baseline, highlighting every difference: No undetected changes.

TestDrive-UAT was used to capture every business process across multiple applications. The result was easy to follow documentation and animations that provided the small automation team with the information they needed.

Equally Qualify has enabled Marc to implement his weekly ‘carousel’ vision. Expected differences are claimed by his business analysts while the abandoned bags, the unexpected changes, are examined to determine if they are defects which must be addressed or pulled before release to the production environment.

The solution would need to check every item on every screen and report all differences

Marc knew they needed to test the areas that had changed, but as importantly, they needed to test that defects had not been unwittingly introduced as a result.Marc likened the process to an airport luggage carousel, where every difference found in a regression test is automatically captured and tracked in the same way that all the baggage from an aircraft arrives on the carousel. Business analysts (just like passengers) would ‘collect’ the differences they expected and anything still on the carousel would be an unexpected change needing investigation

The Results

Over 100 business scenarios now fully automated. No issues reached production

  • First scenarios up and running within 6 weeks

  • 80 complex scenarios completed within 5 months

  •  Several significant defects, which would previously have been missed, have been caught before production

  •  No issues in the areas covered by the automation scenarios have reached production

  • Efficiency, especially for super users, was dramatically increased. No user fatigue associated with conference room pilots

  •  Completely documented test results are also being used as the basis for training and on-boarding new staff. Marc estimates that hundreds of man days have been saved in this area alone

  • CertainTeed can now support a faster delivery schedule. More is now achieved with the same resources, including the automatic creation of a complete audit trail

                                                               

Marc reports that he now has a happier user base who have regained confidence that IT can deliver the evolution needed, together with the stable platform, for business success

The Future

Marc and his team are now building on this success, increasing the number of business scenarios that are automated across their various divisions and expanding the number of applications they cover.