Managing Software Testing

You cannot test quality into software.

Some project managers make the mistake of stepping back when software testing takes the forefront in their projects.

 

It may be due to a lack of hands-on testing experience, concerns over the increasingly technical nature of software testing, or a willingness to let the QA Lead drive the bus for a short period. None of these reasons are valid. Regardless of their roles prior to becoming PMs, they can and should continue to lead the team during testing. In the next few articles, we’ll focus on helping PMs better understand how to guide their teams to delivering better software. Testing the software is just one part of that journey.

 

Let’s clear up some misperceptions. The purpose of testing is not to attempt to break the software, nor to find every possible defect, but to demonstrate that the software will fulfill its intended purpose with a reasonable level of confidence. As highlighted by the quote above, software quality must be built into the project and development process from the beginning rather than being added through testing. Testing allows us to confirm that the expected quality is there and to find and correct those places where it is not. Remember the definition of software quality that we are using:

 

Quality code is code that in order of importance, does what it is supposed to do, is bug free, and is well-crafted.

– Stephen Vance

 

The Institute of Electrical and Electronic Engineers (IEEE) definitions for verification and validation apply during this project phase, so their formal definitions may inform discussions about software performance under testing.

• Verification: The determination by objective, repeatable methods that an item satisfies its stated requirements.*

 

In other words, the software works as it is intended to work, and it meets the requirements it is connected to.

• Validation: The determination by objective, repeatable methods that an item can be used for a specific purpose.*

 

In other words, the software is suitable for purpose, meaning the requirements correctly describe the business need that the software satisfies.

 

(*Adapted from The Project Manager’s Guide to Software Engineering’s Best Practices, by Mark J. Christensen and Richard H. Thayer, IEEE Computer Society Press, Los Alamitos, CA, 2001.)

 

Every test should be traced back to a requirement, either functional or non-functional. If we cannot trace a particular test back to a requirement, we need to question why that test is needed and what purpose it serves. To ensure that we are not wasting time with unnecessary tests and that we are performing the required tests in an appropriate manner and sequence, we start by developing a test strategy. The test strategy defines what will be tested, how it will be tested, and what results are needed to determine that the system is ready for production.

 

Advertisement
[widget id=”custom_html-68″]

 

Once the test strategy is established, one or more test plans are created to meet the goals of that strategy in a methodical, effective, and efficient manner. Test plans describe the purpose of the test, entry and exit conditions, and specifically where and how the system will be tested. Once we have test plans, we can dive into the detailed planning of each test or series of tests. This usually results in test cases, definitions of test environments or regions, test interfaces, test data, and so on.

 

While the project manager is not responsible for creating any of these documents or artifacts, they should be closely involved in their development, review, and ultimate approval. They will have a significant impact on the project schedule, including task duration, task sequencing, key gates, and their location in the schedule. Quality assurance and testing schedules must be coordinated with those of the other teams within the project and typically external to the project (for example, the infrastructure team). These, in turn, impact the budgets and resources that go into the project plan. There may be significant risks or issues that will need to be addressed via specific testing, such as a new external interface or a set of complex calculations, with solutions that must be clearly described in both the testing and overall project plans.

 

The project manager is not expected to be an expert in software testing design or execution, just as they are not expected to be software engineers. They are expected to be familiar with the role, purpose, and types of testing required by their project under the delivery methodology being followed. PMs should ask thoughtful questions, ensure open issues are properly resolved, and participate in walkthroughs of key quality assurance and testing deliverables. We’ll briefly go through common ones in the following sections. While PMs should not be the ones driving the team through testing, they need to ensure that it is being done and done effectively. Some tips on how to do this are covered below.

 

The key point to remember is to be proactive, not passive, with these deliverables and tasks. Although quality cannot be created by testing, testing should identify where quality is lacking so it can be improved. Thorough testing supported by appropriate metrics will find and eliminate most defects and improve confidence that the team is delivering a high-quality system. This section highlights key considerations and leverage points for PMs to help them get the best results possible. There are separate books, whitepapers, and training courses that go into software testing in greater depth. Don’t hesitate to refer to them if desired or if you encounter a particularly unique or difficult testing issue.

 

Next Up: Types of Software Testing

 

Concerned about all the news stories about significant software failures over the past few months? My upcoming book, Building Better Software is focused on providing Project Managers an easy-to-follow guide for successful software development projects.