Vault’s Ultimate Guide: Tracking KPIs for your QA team

From history we’ve learnt that not having adequate testing can lead to epic failures which can result in huge financial damages and losses. NASA's Mars Climate Orbiter Disaster is a famous example which cost them $125 million because one engineering team used metric units while another used imperial units for a key spacecraft operation. Edward Weiler, NASA’s Associate Administrator for Space Science, later explained: “People sometimes make errors. The problem here was not the error, it was the failure of NASA's systems engineering, and the checks and balances in our processes to detect the error. That's why we lost the spacecraft."

You may also remember the iCloud Hack in August 2014 where almost 500 intimate pictures of celebrities got posted on social channels and sites like Reddit. It was later found that it could have been due to a security issue in the iCloud API.

The importance of software testing

While testing is important in every business or field, it has a very special place in software development. A good testing team is something every IT company should consider having or outsourcing. Did you know you could earn more by spending money on software testing before launching a product?

Good software can help boost the productivity and efficiency of your test and QA team, and make you aware of what to measure with tracked KPIs in order to improve your processes. KPIs in the software testing industry are some measurable values that assess the efficiency and effectiveness of the testing process as a whole.

Let’s have a look at some of the key testing indicators which you can easily measure with Vault:

Active defects are all the bugs that aren’t closed yet. With a Kanban view in Vault, you can easily set up which lists with bugs will be tracked with an active status. It can be New, To Do, Planned, Open, Reopened, Fixed, but not verified or other. The QA manager needs to define the threshold value beyond which immediate action will need to be taken to decrease the number of active defects. A test manager determines how many open defects are acceptable to have. The rule is: the lower the number of bugs, the higher the quality of IT projects.

Defects tracked per employee is used to measure the effectiveness of each tester but is subjective as some bugs could be harder to find than others. That’s why defining the complexity level of each task is crucial. More complex defects will usually require a higher level of focus from testers.

Rejected defects measures the percentage of defects that are rejected compared to the total number of defects reported. If the percentage is greater than the set threshold value, then the fundamental problem must be recognized and solved. This could result in more training for software testers.

High/Critical defects are the most serious out of the three defect types and can cause harm to the user or owner of the product. High risk defects may put businesses at serious risk of lawsuits, or product liability issues. Make sure your testing team can evaluate the risk level when setting the severity of a defect. If a defect status is High, throw a red flag.

Accepted defects defines the total number of defects accepted by the development team and can also be measured as a percentage. With this KPI, you can evaluate which tester found the highest number of relevant defects in the system.

Adopt a more effective way of testing and ensure your teams collaborate effortlessly. With Vault, you can create a specific test case run, add test cases, and assign them to your team members. Boost your team’s performance by measuring the time spent on individual cases, tracking the overall execution, and capturing results.

Read on: How a test case management tool can help to your QA team

Test case effectiveness helps to understand the effectiveness of the test case design process and measures the number of defects reported via designed test cases.

Time schedule and constraint is used to measure the average time of test execution. It helps in providing testing time estimates during release planning or while providing the development and testing plans to the project managers.

 Read on: Vault's Ultimate Guide: Tracking KPIs for your software development team

How can software testing save you money?

As mentioned at the start of the blog, software testing and Quality Assurance can save you money. The correlation between them is sometimes hidden but think of it as a precaution.

With bugs and errors timely detected, your product can be launched as planned. If you’re using the Waterfall approach, it can be an expensive mistake not to test and find out one or more of the development phases contained errors. Fixing these mistakes will raise the cost of the whole project and potentially make it worthless.

The good thing is you can now empower your testing team, or the whole process, with a useful tool like Vault. From previous experience we understand that it can be difficult to jump on the “testing train” without dedicated specialists. Be sure to reach out to us and we’ll present you with several options on how to effectively prevent mistakes.