Blog

Chasing 100% code coverage ?

Automated tests are some kind of an “invisible guardian” that is not only great, but also boost your confidence as a developer. When well written, it gives you the freedom to make any changes in your code without fear. You can add features without breaking anything.

I remember this one time project where we made major changes to our backend architecture. Our high-level tests, while unchanged, were guiding us into this refactoring. We didn’t break anything. Not a single bug were added to the list that week. This feeling cannot be described. You cannot make it clear to a project manager how this “invisible guardian” actually improve your confidence and saves you a great amount of time.

I’ve discussed with several developers and one subject always stand out of the crowd: tests. How should I approach them, how can I improve my testing strategy, and so on. While these questions are all interesting, I must admit that there is one that really got my attention recently:

What should be a reasonable code coverage percentage to aim for?

Obviously, you could simply throw out a number and commit to it. We could aim for perfection and shoot for 100%, while we could also be “reasonable” and aim for 80%. In fact, none of these answers make me comfortable. Even though, while almost impossible, my software has 100% code coverage, I still can blindly make changes and break things. How is this possible if all my software is, theoretically, tested?

Answering the question with a number is simply don’t understanding the meaning of automated tests. Why would you blindly aim for a metric? Actually, the question isn’t really about metrics. It’s about confidence. You shouldn’t aim for a percentage level, but rather for a confidence level. How confident are you with your actual system? What would happen if tomorrow I hire a team of 100 QA engineers to test your system? Would you put your career at stake?

The last statement is a bit extreme and wish it doesn’t really happen. We, humans, make mistakes. It happens. But you can actually change your mindset. You should strive to validate the behavior of your features, and not the actual code. Testing the internal state of a class does not give you that high level of confidence you need to iterate quickly. Actually, aiming for a percentage will lead you to create « dummy tests » simply to reach it. You’ll have a false sense of confidence and most likely build a legacy system.

The most beautiful applications will certainly attract consumers, but bugs will repel them. A great look and feel will not compensate a system that doesn’t work. Hence, a high level of confidence will surely be an asset to retain your consumers.

I believe now that your next concern should be on increasing this level of confidence. Feel free to share your thoughts with me.

Share this post

Comment (0)

Leave a comment