The Evil Unit Test

 (This is copied from the blog by Alberto Gutierrez, posted January 27th, 2012 at 7:54 am, on makinggoodsoftware.com, which recently vanished. I pulled this from the Google cache. I largely agree. ~cd)

(Addendum: I had a conversation on this point with Earl Everett, after his engaging talk at Agile Austin. The point of contention was whether an internal class can be considered a "product" to another developer. Think of an algorithm which you will present to others. You will test the algorithm thoroughly, with full branch coverage etc. Do you need to "unit-test" the underlying classes that were used to implement the algorithm?)

The evil unit test.

Written by Alberto Gutierrez

Unit tests can be evil, I know that sounds harsh, but I think there is a battle to be fought to make people understand that unit tests are not always good, and that they actually can become evil, specially when overused.
Think in a foreach, most languages have an implementation of this flow statement. Is the foreach a useful utility for programmers? Definitely! What about function pointers? or Design patterns? or ORM frameworks?… All of them are useful utilities, but can you imagine a programmer, or a book naming the following principles?

• You shall use a foreach every time you need to perform a loop.
• Every third method shall implement a design pattern.
• You will never access your database if you are not using an ORM.

Stupid, right? So why is it that there are programmers happy with statements like:

• Every public method shall have a unit test.
• Unit tests are not to be deleted. There is always a reason for a unit test to exist!
• Unit tests must be created before the code they test.
• Unit test coverage should be 100%.

Unit tests, as any other utility, tool, principle, philosophy etc. are only to be used when applicable and convenient, and not to be taken as religious dogmas. I have already made this point in my previous article, but since it generated so much controversy I thought it would be good to expand on it. If you can’t unit test a piece of code from a behaviour/black box perspective, then you probably shouldn’t write a unit test at all. Your unit test should check what your code is doing, not how. If you create a unit test that it only focus on how your code is, instead of what it does, then you are writing an evil test because:

1. Evil tests create a lock on how the code is implemented.

Every time the implementation of the method changes, even if its behaviour is maintained, the test becomes red. There is no reason to lock the implementation, because there is no advantage on doing so. If there is some code that is especially tricky, locking its implementation with tests won’t make it easier to understand or won’t force anyone to think twice before changing it.

2. Cause duplication.

The beauty of code with good unit tests is that they compliment each other; the code to be tested represents the implementation (the how), and the test guarantees behaviour (the what). On the other hand, evil unit tests look again into the how, so you will have two places where you state how do you think a problem should be solved, only that you will be using different expressions to do so. Is like saying: “2*3 = 2 + 2 + 2”, and then writing a test that says “Guarantee that 2*3 is the addition of 2 three times”, as opposite to “Guarantee that 2*3 is 6”. Testing the implementation and not the behaviour is a waste of time and serves no purpose since you can always go to the code to find what the implementation is.

3. Builds uncertainty on the tests (red is meaningless).

One of the main advantages of having an automated test suite is that feedback loops become shorter. It takes less time from creating a bug to detecting it. When the code gets polluted with bad unit tests, the automated tests seem to start getting broken randomly and this sometimes leads to a point where developers don’t care much anymore about having their tests passing as usual as they should.

4. Decrease productivity.

Given all the circumstances mentioned already, developers dealing with evil unit tests are faced with many inconveniences that forces them to expend time in unproductive tasks, like refactoring useless unit tests, writing them…

5. Discourage change.

Code bloated with bad unit tests is a nightmare to maintain, you make an implementation change, not a behaviour change, and you have to expend ages fixing tests, hence you minimize your implementation changes which is the core idea behind refactoring.
Consider the following principles then:
1. Delete evil tests. There seems to be a taboo around deleting unit tests, is like they are untouchables, well that’s no true, if the test is not adding value, then the test is an evil test and MUST be deleted.
2. Minimise the areas of your code that can’t be effectively unit tested. It is also true that sometimes the problem is that the code is written so that areas that should be unit testable are tied to components that make difficult the unit testing, in this case, the programmers needs to expend more time understanding the principles behind loose coupling and separation of concerns.
3. Write integrations tests for areas that are not unit testable. There is always going to be code in your application that is going to be legitimately not unit testable, for this code, simply don’t write a unit test, write an integration test.