Testing knowledge base

The objective of defects by how found is to show how the defects were discovered, for example, by external customers, manual testing, and the like. If a very low percentage of defects was discovered through inspections, walkthroughs, or JADs, this indicates that there may be too much emphasis on testing and too little on the review process. The percentage differences between manual and automated testing also illustrate the contribution of automated testing to the process.

The objective of root cause analysis is to show the source of the defects, for example, architectural, functional, usability, and so on. If the majority of the defects are architectural, this will pervade the whole system and require a great deal of redesign and rework. High percentage categories should be examined for maintainability after it is released for production.

The objective of test burnout tracking is to indicate the rate of uncovering defects for the entire project and is a valuable test completion indicator. The cumulative (e.g., running total) number of defects and defects by time period help predict when fewer and fewer defects are being discovered. This is indicated when the cumulative curve “bends” and the defects by time period approach zero.

The objective of defect severity status is to show the distribution of the three severity categories for the entire project, for example, critical, major, and minor. A large percentage of defects in the critical category indicate that there existed a problem with the design or architecture of the application, which should be examined for maintainability after it is released for production.

The objective of defect gap analysis is to show the gap between the number of defects that has been uncovered and the number that has been corrected during the entire project. At project completion these curves should coincide, indicating that the majority of the defects uncovered have been corrected and the system is ready for production.

Today, my team and I had Study Group as usually every Wednesday and we were talking about metrics.

You know, there are a lot of metrics that can be used in every project but it didn't used.

Most interesting thing: I did a small survey among my team's member about what metrics did they use and how was it used by them and project team.

Details of survey:
1. Didn't use - 50%
2. Used but there was not significant value from this (ISO methodology were used) - 37.5%
3. Used and it gave real value to the project. - 12.5% 

Amount of people that were take into consideration during survey: 8

So, what does it mean? Many people don't understand what metrics are and how it should be used!

How do you use metrics in your project?