Implementation knowledge for Test Managers

In the arena of Formula one, thousands of hours of work and millions of spend goes into product design, development and testing. Product testing happens not only for the driver to perform but also for a 22 member pit stop crew who has to deliver a 3 second operation, that has a big influence in winning or losing a race. This is no different to an implementation phase of a software project where a tested product lands in the race track, the PROD environment.

There could be several Known unknowns that drive a test strategy during planning phase, one of the important drivers being the Implementation strategy itself. Understanding the implementation approach, hot spots and quality risks and embracing them by incorporating into test strategy as early as possible is the key. It’s a fact that quality issues during implementation phase are not only costly but painful where the development/test teams become lean by then (not by tiredness, but for the purpose!). Let us see why and how testing team can advocate the quality on Implementation, the little known unknown at the time of project planning.

There are Functional and Non Functional requirements that talk about the product’s expected behaviour once it’s up and running but there could be least explained documentation on the product’s journey to the destination. Product quality is experienced by the customer right from the time it goes out of testing shop. The ability of the product to get deployed into prod before 100% live has to be of good quality and needs to be tested with equal importance to the capability of the product. This becomes complex in staged implementations where the product gets rolled out in waves in order to manage the business impact in a controlled manner.

Below are some of the checkpoints to the above context, the testing team can ensure while shaping a test strategy,

• Understand the Implementation strategy – if staged roll out, what are the quality risks at each stage?
• Understand the code drops that leads to system test – what is the success criteria of each code drop, how are they different from implementation stages?
• Understand the NFRs and challenge the same for Implementation specific call outs like data migration, production down times, business thresholds and limitations in live systems, what tasks run in parallel, what cannot?
• Find out the hidden risks - Are there any open questions left by designers and architects to figure out on implementation test, based on system test?
• Who is going to do the actual implementation, have they signed off the implementation strategy?
• Have the Test Environment and Test Data teams understood the implementation strategy?
• Have the Dev teams understood Implementation strategy and aware of any implementation specific deliverables? (e.g.: custom data load tool), does it need to be tested, though it will not be part of the product in Live?

Testing team challenging the project quite early in above mentioned dimensions not only helps reduce cost of poor quality but also facilitates continuous improvements to the solution by making it more effective. Untested or under tested Implementation procedures and tools against the expected behaviour sometimes may pull out product from getting live, leading to a complicated roll back and placing the current system with issues. We really don’t want to see a mechanic struggling with a faulty wheel gun while trying to remove the wheel, or refuelling rigs failing to pump 10 Litres/second, while the full media broadcasting the situation to millions of television viewers around the world. Abandonment of pit stop refuelling since 2009 is indeed a continuous improvement on product safety. Yes, testing team can influence the implementation by asking risk questions quite early in the project and devising a risk mitigation strategy.