Knowledge of variations for a Test Manager

Think of software testing as a system that comprises of processes, people and technology as components. Its essential for a test manager to understand the system and the variations associated to its components. Each system has a level of complexity and emit unique behaviours. When it comes to testing phase, a test manager becomes effective if he understands the variations in the processes so that he can measure, monitor and control the processes to achieve the goal.

Some of the example variations of a typical test phase are,

• What is the ideal defect fix turn around time to be assumed while planning daily run plan? (experience of Dev team, how much they are loaded on other activities are some variations)
• What is the testers effectiveness in finding valid defects? (tester's knowledge of the product under test, how the defect rejection rate looks like in previous cycles are variations)
• What is the expected defect leak level across test cycles? (initial cycles may have high leaks as the team is in the process of gaining knowledge)
• What is ability of test bed to behave like a real solution? (level of stubs, downsized servers, license restrictions - are variations)
• What is the testers daily productivity? (reliability on automation, tester experience, working in night shifts? - are variations)
• What is the psychology of the people? (people morale is a variation, there could be degraded quality of fix or test in last cycles of very large projects due to over stretched teams. On doubt of a defect, a tester may 'Pass' and the other may 'Fail' a test - largely depends on personality)

Without understanding the variations, the goals set on test metrics may accomplish nothing. A goal outside the control limit cannot be accomplished without changing the system. Project management has to be mindful of what the system can produce and set the realistic goals considering variations.

Quality is Free

'Quality is Free' is a popular book written 35 years back by Phil Crosby. I would like to reflect here on the Quality Management Maturity Grid explained by the author. The Maturity Grid discusses the Quality maturity in 5 Stages - Uncertainty -> Awakening -> Enlightenment ->  Wisdom -> Certainty. How an organization moves from Quality Uncertainty to Certainty against dimensions such has Management Understanding/Attitude, Costing metrics, Continuous Improvements etc. The book advocates the Cost of Quality (my favorite topic) and finally summaries that the Cost of Quality is free for an organization if they do the job right the first time against the requirements.

Up for Test Program Management?

Test Program Management is all about managing multiple test projects which are of similar nature (or) projects planned for a common purpose, goal or objective. For someone who does test project management, there will be a question on "how and when should I elevate my self to Test Program Management?".

Here are the skills that is required......benchmark yourself please.

• Should be a seasoned Test Project Manager - should have done end to end test project management with array of testing types across STLC
• Should have the ability to generate programme level test assets that can be applicable for projects across the board
• Should have a mind for organizational views - why the program is needed? what are the program goals? Who are the Internal/External Stakeholders and what is the impact to them etc
• Should have a business mindset - ability to run cost budgeting, knowledge on financial impact to the organization due to program outcome etc
• Should be an Expert in Stakeholder management and Communications - The senior leadership sees the Program Manager as one stop shop for the program health. Any forecast on the program outcome has to be predicted by the program manager and put it through chain of command appropriately
• Most importantly, the Governance skills - able to establish strong governance model to run a large pool of teams seamless

How Downtime tracking will help Test Managers?

Typically, the test environment down times and the associated tester idle time is tracked across the STLC. In fact the 'Environment Availability' forms one of the dependency metrics. When the Test Manager closely monitors this downtime statistics, several risks, rework, cost overruns can be mitigated or planned around.

For instance, if there is a downtime observed due to access blockage on other interfacing systems, there can be service virtualization implemented. A downtime associated with the DB down for backups or for batch runs can be avoided by scheduling the backups or job runs as a nightly run.

The real catch is when it comes to T&M project, where you will tend to bill this downtime but have to negotiate with customers with right rationale!