Archive for the ‘Automated Testing’ Category

Drilling deeper into the World Quality Report

Some good headlines about emerging industry trends came out of the recent World Quality Report produced by HP, Sogeti and Capgemini, but as I read deeper, some things started niggling at me, so much so that I decided they were worthy of a blog post.

For instance:

‘Only 4% of IT professionals agree that their ALM investments are fully paying off, and only slightly more than a third said that half or more of their solutions have been fully implemented and are helping to improve application quality.’ (Pg.8)

HP and Sogeti state that ‘QA organizations need to focus on increasing the adoption rates of their purchased technologies’. This is of course great advice, but both companies have a vested interest in these users throwing more and more money at consultancy, training, and in HP’s case, overpriced and difficult to implement products, (that consequently need a lot of ‘services’ cash thrown at them).

In my opinion it would be more prudent to look in more detail at why these companies are not getting a return on their investments or managing to fully implement their products.

The answers seem to fall into three categories. Company issues, Resource issues and Technology issues, with technology resoundingly winning the prize for biggest stumbling block.

20% failed due to lack of an internal process or support from management. The question does beg to be asked – how on earth did they ever manage to purchase their ALM technologies without some form of internal support? To make ALM successful, it must touch more than just the QA team: support needs to be gained, processes mapped out and business goals and requirements defined way before you make decisions about what technology to buy. It’s really not surprising that these projects failed.

26% stated that not enough resources were invested into the adoption of the technologies. Now I could have classed this under company issues, but I think you really need to look deeper than just writing this off as a staffing issue. It could equally be a technology issue. Was it that not enough staff were trained? Was the project badly planned? Was the technology too complicated for users outside the dev/test team to adopt?

In terms of technology, a whopping 41% struggled because their ALM investment was the wrong choice of technology; it failed to integrate with other technologies or was too complicated and required specialist skills that were thin on the ground. I’ve lost count of the amount of times we hear stories like that when meeting companies across the globe. To truly embed a solution in your company, you need to empower all stakeholders. Unless everyone involved in the delivery of IT projects can collaborate using the solution, it’s just not going to work. When choosing a solution, you need to think about how easily management, business analysts, business users, developers, project managers and testers are able to get what they need out of the solution.

‘Companies prefer testers who have both strong technical skills and relevant domain and business knowledge’ (Pg.11)

Well the stats don’t really allude to that. The question that was asked was – When hiring testers which of the following skills are most important to you? Well obviously QA skills came out tops at 31%. Having a good grounding and understanding of the principles of Quality Assurance is key for testers, I’m actually more surprised that this figure wasn’t higher, but interestingly, the second largest desired skill is business knowledge (22%). This is something we come across time and time again with companies we talk to; so many of them utilise business users for the testing phase. Take SAP testing for example, business process is key. You really need to leverage the knowledge that the business users have about how the system is supposed to perform and exactly how they all use it. So many of the accounts we’ve been into have been literally banging their heads up against a brick wall trying to work out how to capture this knowledge or utilise these testers, knowing that it is impossible with their current toolset – HP is just too cumbersome to get non technical business users to adopt. Development skills 9% and scripting skills 10% are actually rated incredibly low when you consider that the market dominating tools actually force these prerequisites upon QA and make these skills imperative at sites where these traditional tools are embedded.

‘Nearly three quarters of respondents say that they do not follow [common test management methodologies]. Instead, their organisations develop and document their own best practices that are followed in the majority of development and testing projects. (Pg.10)

Different groups in the organisation may adopt their own ‘versions’ of the standard practices, and as a result, the company as a whole is not fully realizing the benefits of standardization, economies of scale, common metrics, unified reporting and asset reusability’

Not all companies are equal and each has different ways of doing business. One size DOES NOT fit all, so surely it is good for the industry that companies develop their own best practices? These companies are just using their brains and working out what best suits their own unique needs and circumstances.

Software vendors should be supporting this very obvious progression of development maturity. Why shouldn’t they be able to all work slightly differently, yet still enjoy the benefits of unified reporting, asset reusability, common metrics etc.? Perhaps the answer lies in the fact that HP hasn’t built its software to be this flexible? Maybe it’s time for the dominant market player in test automation and management to listen to what businesses need rather than telling them how they should be working!

This particular bone of contention was revealed earlier this year with a survey of Application Development Managers back in April 2010. The industry is really crying out for flexibility in the way that tools allow them to work, which is one of the reasons that Original Software developed Qualify, a process and methodology agnostic Application Quality Management solution. Qualify allows businesses to map their own processes, use standard methodology templates, tweak them to suit their own needs and even run multiple methodologies across different teams and projects, with – wait for it, all the added benefits of unified reporting, metrics, re-usability and economies of scale.

Go and check Qualify out.

Why is Test Automation so Cumbersome?

Why is test automation so cumbersome? Original Software has a particular point of view that you may be interested in.

Download our “Throw Away Test Automation” paper today at: http://www.origsoft.com/whitepapers/throw-away-test-automation/

Our view is that traditional test automation systems originated in a world that moved at a much slower pace, where waterfall developments were the only game in town. No-one attempted to tackle fast moving, mission-critical applications, because they knew that the technology simply couldn’t keep up.

These legacy products get their capabilities from powerful scripting languages; something that sounded good in a presentation, but has become a horror in the real world, requiring a cult of high priests (highly skilled and paid test automation engineers) to communicate with the complex and mysterious deity; the test automation tool. Heaven help them if the application changed and the tests needed to be run again!

What companies need in order to overcome the cumbersome nature of test automation, is a solution that makes the art of automation so quick and easy, that they can afford to throw it away if needs be. The biggest barrier of adoption of automation remains the level of maintenance required to sustain it!

Download an Original Insight: “Throw-Away Test Automation”, to explore the issues of test automation and the challenges of implementing automation in an Agile environment.

Download the paper today at: http://www.origsoft.com/whitepapers/throw-away-test-automation/

Thank you very much.

Customers Get Quality-Centric VIP Treatment

Original Software’s robust approach to software quality helps Vermont Information Processing reduce release bottlenecks and improve customer support forecasting

Vermont Information Processing (VIP), IT solutions supplier to the beverage wholesale industry, has utilized Original Software’s end-to-end testing solution to increase its software quality. Even with an intensely accelerated development schedule, VIP has improved its customer upgrade experience and reduced pressure on its support team. This in turn has enabled the business to respond effectively to the complex requirements of its customers during volatile market conditions.

A period of market consolidation had caused significant growing pains for VIP. As its customers’ operations increased in complexity, the software that they relied on to manage their business needed to be able to handle much more complicated processes. VIP’s old development cycle just didn’t fit this new development schedule. Putting out new releases became painful, not just for customers and its development team, but the support team was also beginning to feel the repercussions.

With the help of Original Software, VIP was able to develop a much more robust approach to software quality. The company has completely turned around its QA process and overcome its customers’ cautious attitudes to upgrades. VIP has now transformed into a software testing center of excellence with happy customers who are keen to upgrade to new releases.

Since implementation, VIP’s QA department has increased the scope of test coverage, taking control its test environments, comprehensively unit testing and running full regression tests on all system updates. Previously customers went on average three years before updating systems, creating a bottleneck in the upgrade process, now 97% are running a recent version of the software. The team has also vastly reduced the strain on support. Fixing a single bug could save support around 80 hours; it takes a lot less time to run the regression suite now, which on average turns up 10-12 similar bugs. This equates to five and a half months of support time per release!

VIP moved from having one major update release every three years to producing major releases once a year, with numerous maintenance builds in-between. The old eight month-long beta testing phase was too time consuming for the business with its new rapid development schedule and it could no longer afford to do this. With the crunching development cycle, quality was becoming an issue for VIP. “Our customers were really frustrated with us,” explained Bobby Erwin, Quality Control Analyst at VIP. “To get a new feature, they’d also get all these other problems to boot. This would become a strain on the support team too, as they’d spend hours on the phone going through problems.”

“Our main business drivers were to reduce the level of risk by increasing the scope of test coverage. Original Software was the ideal choice” said Erwin.

VIP purchased the solutions from Original Software in May 2007 and once they’d had the initial training it took two or three months to develop their first regression capabilities. They’ve been developing and building on these tests ever since. “Prior to this we conducted no regression testing at all – just a month or so of acceptance testing by customer service and then the beta tests. Now we have a battery of tests we perform every time we release,” imparted Erwin.

“We are providing a much more robust and better product to the end users,” said Bobby Erwin. “Our support and customer service teams are delighted. Our builds are just so much better now, and we can also tell them exactly how risky a release is, and predict the levels of support that might be needed.”

“The regression tests help us manage our exposure: improving experience for the customer, decreasing support needs for a given update. Testing gives us a feel for how stable the build is. It’s a known entity. If the build looks good in testing, we can install with greater confidence. If we find problems, we get a better idea of the support resources needed,” confirmed Chris Boucher, Support Manager at VIP.

To read the full customer case study, please visit: http://www.origsoft.com/customer-stories/vip

Agile Test Automation, Can It Be Done?

Join us on our next upcomming webinar, “Agile Test Automation, Can It Be Done?”
http://www.origsoft.com/webinars

George Wilson, Original Software, will be hosting the webinar this Wednesday, May 12th 2010, discussing the reality of test automation in an Agile environment.

Agile is a methodology that requires a change in the way QA and development work together. The use of technology and automation are much more difficult and finding a practical approach to testing is critical for successful Agile projects. This webinar will explore how testing in Agile is different and George will give pragmatic advice to ensure application quality.

Listen in and learn how:

*    To overcome the barriers to Agile automation.
*    You can progress into automation gradually and easily even in an Agile environment.
*    Your application quality can be managed with complete visibility.

If you haven’t already registered, make sure you register today at: http://www.origsoft.com/webinars

Disposable test controversy

Search Software Quality has published an article this week by our very own George Wilson.

In his article he talks about the need for more disposable test assets. “In the last decade the rate of business change has risen beyond anything we could have expected. With increasing software development complexity and more and more IT departments taking on an agile approach, traditional test automation has become too cumbersome for most to sustain.”

He argues that “Test automation has failed to date simply because we could not afford to throw it away. Creating any form of automation takes effort and time, when the application under test changes and the automation ceases to work you are faced with a stark choice – either maintain it at additional effort and time or abandon it. If you abandon it you are also writing off the effort and time you invested in creating it, thus bringing the whole concept into question.” (you can read the whole article here)

It’s an interesting angle and certainly a problem that the technology we build at Original Software has been at pains to rectify, however it seems George has touched on a controversial topic. Discussions about the article have sprung up all over the internet and one commenter on a Linked In Group even went as far as saying “There is absolutely no need for ‘throwaway test automation’. Where on EARTH would he get that diagnosis or term?? The whole point of automation is that it doesn’t need to be throwaway.”

We think that the whole point of automation is that it should be flexible enough to adapt so that all your efforts don’t go to waste when applications change, but what about you? Which camp are you in? We’d love to hear your views….

Free Testing Conference in London

Just to let you know, we’ll be in London next week at TestExpo. If anyone is in the area, please come along. It is a great conference and expo for all those involved in testing and QA and best of all, it’s free to attend!

This year’s TestExpo takes place on Thursday (25th March) at the Congress Centre in London.

Our very own George Wilson will be presenting a conference session on the subject of Agile testing. It kicks off at 10am and will explore how to define a test process to ensure application quality within an Agile environment.

Here’s the presentation synopsis – “Agile is a methodology that is seeing increasingly widespread adoption. Yet for the QA professional an Agile approach can cause discomfort. In the ideal world they would have a ‘finished’ product to verify against a finished specification. To be asked to validate a moving target against a changing backdrop is counter intuitive. It means that the use of technology and automation are much more difficult, and it requires a new approach to testing, in the same way that it does for the users and the developers”

Test Automation – Right on Target

This is a presentation given by one of our customers. Ala’a Shouaib at Target Group at the Butler evnt recently

The Seven Deadly Sins of Software Test Automation

Just to get into the spirit of Halloween next week, take a look at our new whitepaper, “The Seven Deadly Sins of Software Test Automation,” (cue evil laugh). We do enjoy being original and I am sure you will have a little chuckle to yourselves with this one!

In this whitepaper we explore each of the seven deadly sins as they relate to software test automation. Instances that Original Software comes across time and time again and traps clients have often fallen into because of their earthly vices.

I don’t want to give too much away,  so if you would like to download this webform free whitepaper, here is the download link http://www.origsoft.com/whitepapers/seven-deadly-sins/

I am quite eager to see what you think of this piece, so please write back with any comments! If you have a great sense of humour, why not reply back with your own software testing deadly sins!

Great Places to network with other Test and QA professionals

(Part 1 – Linked-In)

Linked-In is fast becoming one of the best platforms to pick up industry related news and network with other professionals. There are literally thousands of Linked-In groups, so much so, that finding the most relevant, biggest, best or most active groups can be a bit of a challenge!

To help you on your way, here is a selection of the groups we’re familiar with and can heartily recommend.

Agile (4,813 members)

Agile Enthusiast (638)

Agile Progect Management Group (3,851)

Agile Testing (595)

Agilistias (2,415)

American Society for Quality (11,054)

Automation Testing (518)

Methods & Tools (822)

QA Automation Architect (664)

QA Test Automation (1,010)

QA/Testing (1,193)

QAGuild Network (5,545)

Quality Assurance and Test Professionals / Analysts (242)

Senior Testing Professionals (1,579)

Software Test & Performance Group (914)

Software Testing & QA (997)

Software Testing & Quality Assurance (19,957)

Software Testing and Quality Assurance (2,561)

Software Testing Club (3,705)

SQA forums (2,366)

Test automation (3,875)

Test Development (552)

Test Management Forum (530)

Test Republic (298)

Test strategy and test management (741 members)

NEWS: AppLabs and Original Software Partner to Help Clients Take the Step from Manual to Automated Testing

AppLabs sees huge value proposition for its clients with agile-friendly automation suite

11 August 2009 – Original Software today announced a strategic partnership with AppLabs, the world’s largest software testing and quality management company. AppLabs has entered into an agreement with Original Software to take to market its innovative quality management suite. The agreement will likewise allow Original Software to position AppLabs’ services into its own customer base.

Scott Andress, Vice President, Global Alliances and Service Lines at AppLabs commented: “As a tool agnostic services company, we focus on putting the right solution into the right project. We’ve spotted a compelling value proposition for our clients with Original Software’s suite, which can be a great fit for most agile developments, can speed up time-to-market and can provide a silver bullet for those clients struggling to make the transition from manual to automated testing.”

The industry analysts agree. A recent Ovum report stated: ‘TestDrive-Assist provides you with a rich environment for manual testing, but the bigger payback is how it facilitates adoption of full automation. Uniquely for any of the guided manual test execution products that we know of, Original can automatically convert a manual TestDrive-Assist test into a fully automated TestDrive test that can be repeated.” “Overall this is a low risk, low cost way of progressively adopting test automation.” “It fits in with classic waterfall processes, and it can also be used in agile processes more easily than other functional test automation suites.”

“TestDrive-Assist cuts the time it takes to manually test in half, with some customers having recorded savings of 75-80 percent” said Colin Armitage, CEO of Original Software. He continued: “This partnership not only provides us with a strong route to market, but allows us to add services to our expanding portfolio. AppLabs’ expertise in testing services and its global, skilled resource pool will be a valued addition to our offering, giving customers the option to outsource projects or business functions or take advantage of specialized consultancy to optimize their IT and business quality management and testing capabilities.”