Archive for the ‘Test Planning and requirements’ Category
Are HP QTP & Quality Center the emperor’s new clothes?
It may be a tale from our childhood but I can think of no better analogy for the current state of test management and test automation tools market: The Emperor’s New Clothes. Now for those of you who cannot recall the story let’s have a quick recap.
The Emperor’s New Clothes
The Emperor by definition was a powerful chap and one who wanted acclamation and praise from his court. So when a couple of con-men pitched up at court promising His Highness the ultimate in designer fashion they found an eager audience in the main man. Now neither conman knew much about tailoring so instead they convinced the Emperor that a non-existent figment of their imagination they held in their arms, was in fact the finest suit ever made, un-equalled in all of the kingdom. The Emperor fell for their pitch hook, line and sinker. So convinced was he of its beauty that he paraded himself to his court. Sadly the court, being so used to only saying the things he wanted to hear, was emasculated and no-one had the gumption to speak up.
Things did change when the Emperor decided to parade himself through the city but by this time the con-men were long gone.
So what’s this got to do with testing?
Let’s consider what we want from our tools. What does the ultimate ready to wear, waterproof, uncrushable and debonair tool-set look like?
1. Every project starts with a plan and that plan will be the backbone of the project. But every project is different and a good tool can adapt to every approach utilized. Waterfall hasn’t gone away and for some companies it never will. However the converse is not true. Pretty much every company we know has embraced agile methodologies to a greater or lesser extent. And the agile world is fluid rather than static. Teams look to refine their agile approach based on their experiences and evolving industry best practice. So your application quality management platform needs to support multiple concurrent methodologies with the ability to consolidate common data. Now ask yourself if your current tool can do that. If you are starting to see the proliferation of multiple tools each with the same objective then you already know the answer.
2. Much of the testing will be manual. Much of the manual testing will be done by power users from the line of business who can ill afford the time you demand. Shouldn’t a tool set make manual testing fast, to minimize the impact on everyone involved and to capture the business knowledge to lessen the burden in future projects?
3. And when it comes to test automation, be brave, take a deep breath and very quietly repeat “faster, better, cheaper”. Go on, try it again. Now be really brave and ask yourself whether the automation tools you use are delivering on that mantra. Slow, costly, fragile and ill-suited to agile developments are phrases that may come to mind instead.
So there you go. If you look at what you need and you compare it with what you’ve got I think you’ll find you’re as naked as the day you were born.
You can read more about an alternative to HP QTP or QC here.
HMRC Software Glitch Makes Returns Taxing
Yesterday, January 31st, was the last chance for self-assessors to submit their tax return online at the website for Her Majesty’s Revenue and Customs (HMRC). Even though the HMRC said only on Friday that their computer system was working well and they did not foresee the problems experienced in recent years, our majesty’s government failed to test thoroughly again!!
Apparently, accountants were having problems after it appeared that the company that supplies software which allows accountancy firms and advisors to access the HMRC website had shut its links early. Returns that should have taken no more than 10 minutes to file, were taking up to an hour. HMRC confirmed that some advisors using third party software were experiencing what they called a ‘slowdown’.
CCH Personal Tax was the software concerned and although its parent company, Wolters Kluwer, said they had not been notified of any issues with the software, problems certainly existed.
Now we can’t be sure what the problems really were, but this story is yet another example of poor or incomplete testing. Such high profile software glitches only go to justify Original Software’s mantra that “application quality must become a business imperative” – especially if government reputations are to remain unscathed.
Source: Yahoo News
ERP Testing Saps Resource at Marstons
When Judy Doust at Marston’s told us about her pain points with testing SAP applications, it was obvious that our unique approach to testing would be beneficial. So we put our developers to the task of coming up with an SAP solution. We’re pleased to say they’ve come up trumps and have helped the independent UK brewer and pub retailer to ease the increasing burden of SAP testing without the need for employing additional technical resources…
Historically Marston’s had only been using two SAP modules: Financials and Sales & Distribution, yet it would take over six months to plan and execute the testing of each upgrade. Now the company is using additional modules within the SAP ERP Central Component solution, CRM and SRM within the SAP Business Suite, SAP BI, bespoke applications and many web applications. This increased complexity had become a growing concern for Marston’s IT department. Continuous manual testing was not a feasible option moving forward, especially with the way SAP was planning updates in its latest release. Instead of one big upgrade, patches were to be delivered in regular packages throughout the year. For Marston’s the scope and complexity of testing SAP updates has dramatically increased.
In addition to the long testing cycles, Marston’s also felt it was constrained by a lack of choice in the SAP automated testing market and required the solution it eventually purchased to fit in with how Marston’s ran its business. Marston’s has always utilised the business users as its testers, as it is they who really know how the applications are supposed to work. The business users have no IT skills, but have a more holistic view of the business processes under test. As such, the solution had to be easy to use without the need for any technical knowledge such as programming.
One choice would have been HP, given its monopoly-like relationship with SAP for testing. But HP was not a realistic choice for Marston’s.
Judy Doust, Test Manager at Marston’s, explains: “Our SAP testing is business process driven and as such requires people with knowledge of the business process to actually design and execute tests. Although the integration exists between HP and SAP, I really didn’t like feeling as though I was being tied into something that was not right for our business. HP is a technical solution and I didn’t want to employ another couple of people to do the coding and maintenance, so I went looking for a viable alternative.”
“Our prime objective was not to employ a huge test department. Our end users are the business users – we’ve always used them as testers and it works well for us – it saves on re-work. A system that requires IT specialism would force us to employ a team of specialist testers. We’d still need to utilise the users for the business process knowledge and there would be crossover and duplicated effort. Why perform all that work if there was another option out there?” continued Judy.
When Marston’s learnt about Original Software and realised how easy the solution was in terms of usability, it immediately saw the benefits it could provide the business with its SAP testing. Judy added: “When I approached Original Software to investigate the solution suite’s suitability for SAP testing, their display of willingness and intent gave us confidence.”
“We have now implemented their manual and automated testing solution and are in the process of training users. The beauty of TestDrive-Assist is the non-intrusive way it records activity while the user is testing. This allows us to capture the day-to-day processes of the business users so that their knowledge is retained and shared within our organisation. We will be able to leverage that business process knowledge in order to build automated test regression packs, providing us with huge time savings each time we re-run the tests.”
Marston’s next objective is upgrade regression pack building. “We want to be able to automate and make upgrades less painful, but quality really is at the heart of everything we are doing. We want to improve the quality of our solutions, while at the same time maintain the same level of service for users. Our overall goal is that the business continues to believe in IT and what we are doing – all the way from the business process owners and end users, right up to board director level,” Judy concluded.
Drilling deeper into the World Quality Report
Some good headlines about emerging industry trends came out of the recent World Quality Report produced by HP, Sogeti and Capgemini, but as I read deeper, some things started niggling at me, so much so that I decided they were worthy of a blog post.
‘Only 4% of IT professionals agree that their ALM investments are fully paying off, and only slightly more than a third said that half or more of their solutions have been fully implemented and are helping to improve application quality.’ (Pg.8)
HP and Sogeti state that ‘QA organizations need to focus on increasing the adoption rates of their purchased technologies’. This is of course great advice, but both companies have a vested interest in these users throwing more and more money at consultancy, training, and in HP’s case, overpriced and difficult to implement products, (that consequently need a lot of ‘services’ cash thrown at them).
In my opinion it would be more prudent to look in more detail at why these companies are not getting a return on their investments or managing to fully implement their products.
The answers seem to fall into three categories. Company issues, Resource issues and Technology issues, with technology resoundingly winning the prize for biggest stumbling block.
20% failed due to lack of an internal process or support from management. The question does beg to be asked – how on earth did they ever manage to purchase their ALM technologies without some form of internal support? To make ALM successful, it must touch more than just the QA team: support needs to be gained, processes mapped out and business goals and requirements defined way before you make decisions about what technology to buy. It’s really not surprising that these projects failed.
26% stated that not enough resources were invested into the adoption of the technologies. Now I could have classed this under company issues, but I think you really need to look deeper than just writing this off as a staffing issue. It could equally be a technology issue. Was it that not enough staff were trained? Was the project badly planned? Was the technology too complicated for users outside the dev/test team to adopt?
In terms of technology, a whopping 41% struggled because their ALM investment was the wrong choice of technology; it failed to integrate with other technologies or was too complicated and required specialist skills that were thin on the ground. I’ve lost count of the amount of times we hear stories like that when meeting companies across the globe. To truly embed a solution in your company, you need to empower all stakeholders. Unless everyone involved in the delivery of IT projects can collaborate using the solution, it’s just not going to work. When choosing a solution, you need to think about how easily management, business analysts, business users, developers, project managers and testers are able to get what they need out of the solution.
‘Companies prefer testers who have both strong technical skills and relevant domain and business knowledge’ (Pg.11)
Well the stats don’t really allude to that. The question that was asked was – When hiring testers which of the following skills are most important to you? Well obviously QA skills came out tops at 31%. Having a good grounding and understanding of the principles of Quality Assurance is key for testers, I’m actually more surprised that this figure wasn’t higher, but interestingly, the second largest desired skill is business knowledge (22%). This is something we come across time and time again with companies we talk to; so many of them utilise business users for the testing phase. Take SAP testing for example, business process is key. You really need to leverage the knowledge that the business users have about how the system is supposed to perform and exactly how they all use it. So many of the accounts we’ve been into have been literally banging their heads up against a brick wall trying to work out how to capture this knowledge or utilise these testers, knowing that it is impossible with their current toolset – HP is just too cumbersome to get non technical business users to adopt. Development skills 9% and scripting skills 10% are actually rated incredibly low when you consider that the market dominating tools actually force these prerequisites upon QA and make these skills imperative at sites where these traditional tools are embedded.
‘Nearly three quarters of respondents say that they do not follow [common test management methodologies]. Instead, their organisations develop and document their own best practices that are followed in the majority of development and testing projects. (Pg.10)
Different groups in the organisation may adopt their own ‘versions’ of the standard practices, and as a result, the company as a whole is not fully realizing the benefits of standardization, economies of scale, common metrics, unified reporting and asset reusability’
Not all companies are equal and each has different ways of doing business. One size DOES NOT fit all, so surely it is good for the industry that companies develop their own best practices? These companies are just using their brains and working out what best suits their own unique needs and circumstances.
Software vendors should be supporting this very obvious progression of development maturity. Why shouldn’t they be able to all work slightly differently, yet still enjoy the benefits of unified reporting, asset reusability, common metrics etc.? Perhaps the answer lies in the fact that HP hasn’t built its software to be this flexible? Maybe it’s time for the dominant market player in test automation and management to listen to what businesses need rather than telling them how they should be working!
This particular bone of contention was revealed earlier this year with a survey of Application Development Managers back in April 2010. The industry is really crying out for flexibility in the way that tools allow them to work, which is one of the reasons that Original Software developed Qualify, a process and methodology agnostic Application Quality Management solution. Qualify allows businesses to map their own processes, use standard methodology templates, tweak them to suit their own needs and even run multiple methodologies across different teams and projects, with – wait for it, all the added benefits of unified reporting, metrics, re-usability and economies of scale.
Go and check Qualify out.
Blasting away market frustrations
Today we are officially launching Qualify to the world. Our brand new Quality Management solution aids business agility and unifies the software delivery process.
A recent independent report highlighted a huge amount of discontent in the market, with 84% of those surveyed stating that current products did not meet their functional requirements. Built with the purpose of directly answering these needs, Qualify unites all aspects of the software development lifecycle from requirements through to release. It is also fully integrated with Original Software’s manual and automated test execution solutions for the user interface and database layers.
Qualify encompasses requirements, planning, scheduling, resource management, test execution, defect management and reporting in one platform, ensuring collaboration and communication throughout. The methodology agnostic, entirely code free Quality Management solution, even comes complete with a smart phone interface, allowing instant access to cross-project information from just about anywhere.
Businesses now have a much better option than Quality Center from HP. One that really answers today’s business needs. Colin our CEO had this to say about the launch - “With more complexity in IT projects and a need to respond faster to changing markets, development teams have had to adapt the way they work, requiring processes and technology to support the dynamic nature of their businesses. The absence of a holistic solution for ensuring quality when managing the Software Development Life Cycle has been a bone of contention with the customers we speak with.”
We really think it’s quite brilliant and hope you do to. Check out our brand new Qualify page http://www.origsoft.com/qualify, where you will find links to some great resources including datasheets, a comparison document, analyst reports on Qualify and a new whitepaper with a great Star Wars theme entitled ‘What Yoda can teach us about Quality Management’
NEWS: AppLabs and Original Software Partner to Help Clients Take the Step from Manual to Automated Testing
AppLabs sees huge value proposition for its clients with agile-friendly automation suite
11 August 2009 – Original Software today announced a strategic partnership with AppLabs, the world’s largest software testing and quality management company. AppLabs has entered into an agreement with Original Software to take to market its innovative quality management suite. The agreement will likewise allow Original Software to position AppLabs’ services into its own customer base.
Scott Andress, Vice President, Global Alliances and Service Lines at AppLabs commented: “As a tool agnostic services company, we focus on putting the right solution into the right project. We’ve spotted a compelling value proposition for our clients with Original Software’s suite, which can be a great fit for most agile developments, can speed up time-to-market and can provide a silver bullet for those clients struggling to make the transition from manual to automated testing.”
The industry analysts agree. A recent Ovum report stated: ‘TestDrive-Assist provides you with a rich environment for manual testing, but the bigger payback is how it facilitates adoption of full automation. Uniquely for any of the guided manual test execution products that we know of, Original can automatically convert a manual TestDrive-Assist test into a fully automated TestDrive test that can be repeated.” “Overall this is a low risk, low cost way of progressively adopting test automation.” “It fits in with classic waterfall processes, and it can also be used in agile processes more easily than other functional test automation suites.”
“TestDrive-Assist cuts the time it takes to manually test in half, with some customers having recorded savings of 75-80 percent” said Colin Armitage, CEO of Original Software. He continued: “This partnership not only provides us with a strong route to market, but allows us to add services to our expanding portfolio. AppLabs’ expertise in testing services and its global, skilled resource pool will be a valued addition to our offering, giving customers the option to outsource projects or business functions or take advantage of specialized consultancy to optimize their IT and business quality management and testing capabilities.”
NEWS: Original Software Positioned as a Visionary in Magic Quadrant for Integrated Software Quality Suites
Original Software, the Application Quality Management (AQM) vendor, today announced that it has been placed in the ‘visionaries’ quadrant by leading analyst firm Gartner, Inc. in its recently published Magic Quadrant for Integrated Software Quality Suites, 31 July 2009.
“The total market for test management and functional and load/stress automation is currently valued at just under $1.2 billion, and is growing at more than 8% per year.” said Thomas Murphy in the report1. “Testing software can be an expensive process, but poor software quality leads to user dissatisfaction, as well as increased development and maintenance. Therefore, having a well-defined set of tools and practices to drive software quality will positively affect the overall business bottom line.”
The report evaluates fourteen vendors of software quality solutions based on a rigorous set of criteria that comprise ‘completeness of vision’ and ‘ability to execute’. Original Software has been positioned as a visionary.
Colin Armitage, CEO of Original Software, said: “We consider our positioning in the visionaries quadrant to be confirmation of the real value that our approach to solving challenges in the quality process has brought to our users, extending the reach of these solutions within an organization and providing an early return on investment. We are now expanding on the unique work we have done, in reducing script maintenance, supporting manual testing, code-free automation and test data management, with the launch of a radical new ALM solution later this year.”
This report has recently been published by Gartner. More information can be found on the analyst page of Original Software’s website.
NEWS: Original Software offers ‘Escape’ for apprehensive Borland and Compuware customers
Original Software, the Application Quality Management (AQM) vendor, today announced a swap-out promotion for customers of the soon to be engulfed companies, who face uncertainty in the future of their legacy product roadmaps.
Original Software offers market-leading AQM solutions that span the application development life-cycle from requirements, planning and management to complete automated regression testing, including unique solutions for manual testing and test data management.
“Colin Armitage, CEO of Original Software, said: “Any customer on a current maintenance contract with Borland or Compuware can qualify for the Escape program and swap to one of our comparable solutions, provided they switch the maintenance agreement.” The promotion, which runs until the end of the year, includes preferential pricing on all the company’s other complimentary solutions.
“While MicroFocus has now completed the acquisitions of Borland and Compuware, the future for their customers is far from clear. It is inevitable that one of the solutions will take precedence, leaving the other customer base facing an unpleasant choice. You have to feel special sympathy for the users of the Borland testing products for who this is the second change of ownership in less than three years.”
“The great thing about the Escape program is that companies can remove the uncertainty about products futures and join a company whose vision has been widely recognized by industry analysts like Gartner and Ovum. At the same time they can benefit from great customer service and a modern AQM solution suite that addresses traditional automation and the now widely recognized needs for collaboration, manual test support and test data management”.
RIP TestPlan, Long live Qualify
PRODUCT UPDATE – From next month Original Software will be replacing TestPlan, its original test planning tool, with a brand new solution named Qualify. Although the product will not have its official media launch until Autumn, when the additional enhancements will push it into a category of its own, Qualify v.1 will be on sale from April and far outstrips its predecessor in terms of its configurability and graphical interface.
A complete Application Quality Management (AQM) solution that unites requirements, test execution, defect management and reporting within one platform, with integrated communications channels and multi format reporting, Qualify provides all the information needed to make truly informed application quality decisions.
For more information, please visit the Qualify page on our corporate website , which includes a link to the official data sheet pdf.