Are too many units tests bad for your project?

Perhaps to some this might sound like heresy and to others it my seem like common sense.

I am a fan of unit testing but have recently been thinking about the right number of tests prompted from observing an over zealous attempt at creating unit tests.

During this time I was reminded that tests are not free, they take time to create, maintain and run. The idea is this time is recouped during development by catching errors early and acting as a sanity check. This kind of works out quite well most of the time.

There are times when it can be a drag.

The first of these is small prototype projects. The sort of ones I talk about on this blog. Things that are knocked out on a weekend or at most a month. They are mostly for personal consumption, occasional I do do unit tests but often only start adding unit tests after the first release. If I start on a second iteration it probably means I will work on it on and off for an extended period and unit tests will pay for themselves. I doubt people will disagree with this one as it is not exactly controversial.

The point of this post is I recently observed someone create many hundreds of unit tests that were all permutations of each other. It took him quite a while. All these permutation represent  different options that can be given to a system. The code is programmed such that there are many similar functions (this part of the code if old and by that I mean decades) so he used that as the reasoning for writing the test while altering the code.

The trouble is the possible combinations is so large even this guys herculean attempt could not cover everything and general testing revealed more bugs.

This led me to thinking someone sitting down and attempting to write lots of unit tests should trigger warning alarms. Personally I felt it is more appropriate to spend time going over the code to reducing the number of branches/functions in the system meaning we could cover the code with less unit test.

To answer the obvious re-action. there are many layers of testing so it is possible to refactor code fairly safely without unit tests in place.

TL/DR Sometimes it is better to simplify code rather then write unit test for complex code.