JoshSweeney

The “What is code coverage” cop-out

At one point in time I was a quality assurance engineer and had the pleasure of attending many conferences about testing methodology and how to test throughout the SDLC. In having this education I have always attempted to pass testing information along that I felt would benefit another company or person.

After talking with many developers I have found one thing that is an absolute cop-out.

It starts like this:

Developer: They say we have to many bugs

Me: What is you code coverage?

Developer: Well that depends on what you consider code coverage.

This final statement generally translates to “We have nothing in place to measure code coverage.” In even worse conditions you can append “We write code not tests. Testing is for QA” on to the end.

I know that all of the developers out there now want to kill me. Some say that the “what is code coverage” argument is valid. To that I would say that you are 100% correct. It is a valid question if you have not decided to start testing on the developers side of the house. The fact that you are still avoiding answering the question with a percentage of code coverage instead of this counter question just backs up my point.

To other non-developers including QA and management the issue doesn’t come down to what code coverage is. What they want to see is that the people writing the code are taking every step necessary to test the code that they write and make sure it is the best quality possible before being pushed to QA and then to the world. No matter what you or the analysis tool considers code coverage the rest of the team wants to see metrics. They want coverage represented in a percentage that is moving in the correct direction with every cycle, closer to 100%.

Oddly enough I have found a direct relation to the answer to “what is your code coverage” and the quality of products. Not in every case, but in most cases if you ask the lead developer about their code coverage and they answer with a percentage or metric then you can at a minimum be assured that testing is on their radar. If the number or metric is high then it is likely that the level of quality is higher than to other competing software.

Code coverage is not an exact science and each tool calculates in a different fashion. As long as you or your developers are using the same tool and testing methods then the coverage report should look better with every development cycle. Remember, testing is a process and not a department.

Tags: code coverage