Sense of unit tests without TDD

I have a relevant anecdote from something that's going on right now for me. I'm on a project that does not use TDD. Our QA folks are moving us in that direction, but we're a small outfit and it has been a long, drawn-out process.

Anyways, I was recently using a third-party library to do a specific task. There was an issue regarding the use of that library, so it's been put on me to essentially write a version of that same library on my own. In total, it ended up being about 5,000 lines of executable code and about 2 months of my time. I know lines of code is a poor metric, but for this answer I feel it's a decent indicator of magnitude.

There was one particular data structure I needed which would allow me to keep track of an arbitrary number of bits. Since the project is in Java, I chose Java's BitSet and modified it a bit(I needed the ability to track the leading 0s as well, which Java's BitSet doesn't do for some reason.....). After reaching ~93% coverage I started writing some tests that would actually stress the system I had written. I needed to benchmark certain aspects of the functionality to ensure they would be fast enough for my end requirements. Unsurprisingly, one of the functions I had overridden from the BitSet interface was absurdly slow when dealing with large bit sets(hundreds of millions of bits in this case). Other overridden functions relied on this one function, so it was a huge bottle neck.

What I ended up doing was going to the drawing board, and figuring out a way to manipulate the underlying structure of BitSet, which is a long[]. I designed the algorithm, asked colleagues for their input, and then I set about writing the code. Then, I ran the unit tests. Some of them broke, and the ones that did pointed me exactly to where I needed to look in my algorithm in order to fix it. After fixing all of the errors from the unit tests, I was able to say that the function works as it should. At the very least, I could be as confident that this new algorithm worked as well as the previous algorithm.

Of course, this is not bullet proof. If there's a bug in my code that the unit tests aren't checking for, then I won't know it. But of course, that exact same bug could have been in my slower algorithm as well. However, I can say with a high degree of confidence that I don't have to worry about the wrong output from that particular function. Pre-existing unit tests saved me hours, perhaps days, of trying to test the new algorithm to ensure it was correct.

That is the point of having unit tests regardless of TDD - that is to say, unit tests will do this for you in TDD and outside of TDD all the same, when you end up refactoring/maintaining the code. Of course, this should be paired with regular regression testing, smoke testing, fuzzy testing, etc, but unit testing, as the name states, tests things on the smallest, atomic level possible, which gives you direction on where errors have popped up.

In my case, without the existing unit tests, I would somehow have to come up with a method of ensuring the algorithm works all of the time. Which, in the end...sounds a lot like unit testing, doesn't it?

You can break code roughly into 4 categories:

1. Simple and rarely changes.
2. Simple and frequently changes.
3. Complex and rarely changes.
4. Complex and frequently changes.

Unit tests become more valuable (likely to catch important bugs) the further down the list you go. In my personal projects, I almost always do TDD on category 4. On category 3 I usually do TDD unless manual testing is simpler and faster. For example, antialiasing code would be complex to write, but much easier to verify visually than writing a unit test, so the unit test would only be worth it to me if that code changed frequently. The rest of my code I only put under unit test after I find a bug in that function.

It's sometimes difficult to know beforehand what category a certain block of code fits into. The value of TDD is you don't accidentally miss any of the complex unit tests. The cost of TDD is all the time you spend writing the simple unit tests. However, usually people experienced with a project know with a reasonable degree of certainty what category different parts of code fit into. If you aren't doing TDD, you should at least try to write the most valuable tests.