Tell me more ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.
up vote 16 down vote favorite
1

I was reading this blog by Joel Spolsky about 12 steps to better code. The absence of Test Driven Development really surprised me. So I want to throw the question to the Gurus. Is TDD not really worth the effort?

share|improve this question
12  
That article was written Wednesday, August 09, 2000 (about 12 years ago). Not that TDD wasn't around at that time but I don't believe it enjoyed nearly the buzz that it does these days. – Mike Mar 4 at 17:41
11  
The Joel test is just a set of generic guidelines. Not everything that's "worth the effort" can fit in there. – Yannis Rizos♦ Mar 4 at 17:42
2  
'I've come up with my own, highly irresponsible, sloppy test to rate the quality of a software team. The great part about it is that it takes about 3 minutes... The neat thing about The Joel Test is that it's easy to get a quick yes or no to each question. You don't have to figure out lines-of-code-per-day or average-bugs-per-inflection-point...' - deciding on whether your project will benefit of TDD would take more than 3 minutes and, well, might require figuring average-bugs-per-inflection-point - that's why it's not in the list – gnat Mar 4 at 17:58
Move to Joel Stack plz. It's an interesting q. – Erik Reppen Mar 5 at 4:59
You should consider accepting the answer that links to and quotes directly from Joel, since it doesn't get any more authoritative than that. See programmers.stackexchange.com/a/189493/6586 – Bryan Oakley Mar 7 at 0:18

4 Answers

up vote 27 down vote accepted

Test driven development was virtually unknown before Kent Beck's book came out in 2002, two years after Joel wrote that post. The question then becomes why hasn't Joel updated his test, or if TDD had been better known in 2000 would he have included it among his criteria?

I believe he wouldn't have, for the simple reason that the important thing is you have a well-defined process, not the specific details of that process. It's the same reason he recommends version control without specifying a specific version control system, or recommends having a bug database without recommending a specific brand. Good teams continually improve and adapt, and use tools and processes that are a good fit for their particular situation at that particular time. For some teams, that definitely means TDD. For other teams, not so much. If you do adopt TDD, make sure it's not out of a cargo cult mentality.

share|improve this answer
1  
Also... oh you somewhat hit the TDD is Kool Aid point did't you? – Erik Reppen Mar 5 at 5:00
up vote 5 down vote

Joel Spolsky himself answered this question back in 2009:

Joel: There's a debate over Test Driven Development... should you have unit tests for everything, that kind of stuff... a lot of people write to me, after reading The Joel Test, to say, "You should have a 13th thing on here: Unit Testing, 100% unit tests of all your code."

And that strikes me as being just a little bit too doctrinaire about something that you may not need. Like, the whole idea of agile programming is not to do things before you need them, but to page-fault them in as needed. I feel like automated testing of everything, a lot of times, is just not going to help you. In other words, you're going to write an awful lot of unit tests to insure that code that, really, is going to work, and you're definitely going to find out if it doesn't work [if you don't write the tests] does, actually still work, ... I don't know, I'm going to get such flame mail for this because I'm not expressing it that well. But, I feel like if a team really did have 100% code coverage of their unit tests, there'd be a couple of problems. One, they would have spent an awful lot of time writing unit tests, and they wouldn't necessarily be able to pay for that time in improved quality. I mean, they'd have some improved quality, and they'd have the ability to change things in their code with the confidence that they don't break anything, but that's it.

But the real problem with unit tests as I've discovered is that the type of changes that you tend to make as code evolves tend to break a constant percentage of your unit tests. Sometimes you will make a change to your code that, somehow, breaks 10% of your unit tests. Intentionally. Because you've changed the design of something... you've moved a menu, and now everything that relied on that menu being there... the menu is now elsewhere. And so all those tests now break. And you have to be able to go in and recreate those tests to reflect the new reality of the code.

So the end result is that, as your project gets bigger and bigger, if you really have a lot of unit tests, the amount of investment you'll have to make in maintaining those unit tests, keeping them up-to-date and keeping them passing, starts to become disproportional to the amount of benefit that you get out of them.

share|improve this answer
Really? Downvotes on posting Joel's own answer to the OP's question? – Ross Patterson Mar 6 at 22:37
Hard to figure. Some people use the vote to mean "I approve" rather than "this is useful". This obviously should be the accepted answer because it is definitive. – Bryan Oakley Mar 7 at 0:16
up vote 19 down vote

Joel has actually addressed this specifically in a few places.

He's explained that the things tests are not capable of catching a lot of important issues, particularly subjective ones such as "does this software's user interface suck?" According to him, over-reliance on automated tests at Microsoft is how we ended up with Windows Vista.

He's written how, in his experience, the kinds of bugs that users actually find tend to fall into two categories: 1) the ones that show up in common usage, which the programmers would have found themselves had they run their own code before using it, or 2) edge cases so obscure that no one would have thought to write tests to cover them in the first place. He's stated that only a very small percentage of the bugs he and his team fixes in FogBugz are the sort of thing that unit testing would have caught. (I can't find that article now, but if anyone knows which one I mean, feel free to edit the link into here.)

And he's written how it can be more trouble than it's worth, especially when your project grows into a very large project with many unit tests, and then you change something (intentionally) and end up with a very large number of broken unit tests. He specifically uses the problems that unit testing can cause as the reason why he has not added it as a 13th point to the Joel Test, even when people suggest that he ought to.

share|improve this answer
2  
Actually, you're right. Joel's usual MO is the straw man. Like TDD wouldn't have caught any bugs for me, so it can't be good. Which somewhat misses the point that TDD is not about testing, it's about design. The tests left behind are a bonus. Or to argue that a small change will break many unit tests, which suggests he's just doing it wrong. Or to completely rewrite a SOLID principle before attacking it. That kind of thing. It's actually his proponents that use the circular logic, not him. – pdr Mar 4 at 21:57
3  
I completely agree with these comments of Joel's. I think an even bigger issue is the language--with many dynamic languages I can't imagine doing anything without a unit tests--how else can you tell if a simple typo will cause some problem that you won't see until a critical moment? In statically typed, compiled languages designed to reduce errors you are guided away from all the simplest errors and are left mostly with logic errors. This reduces the need for the type of full-coverage provided by TDD. – Bill K Mar 4 at 22:09
2  
@MasonWheeler: You are seriously arguing that compiler-/type-safety removes the need for unit tests? You are also wilfully ignoring the design benefits of TDD but, more importantly, you must have a hell of a time refactoring anything. Rather, the opposite has been seen to be true: eg .NET developers who follow a TDD methodology suddenly finding themselves frustrated by the amount of code they need to write to please a compiler which is increasingly unhelpful in return. – pdr Mar 4 at 22:20
1  
@pdr: I am seriously arguing that "the need for unit tests" in the first place is the lack of type safety. And, not being a .NET developer, I can't really say much about their experiences, but in my own experience I've found that difficulty in refactoring is based entirely on two factors: whether or not I wrote the code in the first place, and whether or not the author wrote the code well. (Note: points 1 and 2 are not necessarily strongly correlated with each other!) – Mason Wheeler Mar 4 at 22:36
2  
@Pdr Unit tests don't prove your code, they are mostly a syntax checker, but can be very useful during development. Integration and System tests make a lot more sense though. Also most refactorings in statically typed languages can be proven safe, in fact that's what a refactoring is--a set of known "Safe" operations that transform your code without introducing changes. In a static language, the IDE can often make these changes for you and assure that they are safe, something often impossible in dynamic languages which therefore require unit tests to assist with (not prove) the same safety – Bill K Mar 4 at 23:43
show 10 more comments
up vote 5 down vote

No one but Joel could answer that for sure. But we can try some reasons/observations.

First of all, testing is not absent from the Joel's Test

  • Tests are mentioned two times in 12 steps directly (10 and 12)
  • The existence of a build is one of the first points. The idea of having build is to get the capacity to see if they break, so we are (also) talking about testing here.

Secondly, the whole idea of the Joel Test (as I understand it) is to have quick, yes-no questions. "Do you do TDD ?" will not exactly fit in (answer could be : "some of us", "for that part of the code" or "we do unit test".

Thirdly, I think no one said that (even Joel) that those points where "the only ones worth time" (by the way, "do you program" is not on it), just that those are good quick questions to ask when coming into contact with a software team, whether as a future team member or even as a customer - this is a list I gave to some non technical people around me that were looking for clues about how good/bad their own IT department was. It is not everything, but it is really bad to beat in three minutes.

share|improve this answer
3  
"Do you do TDD ?" certainly fits as a yes-no question. And by that I mean it's a question that everyone answers with an emphatic "yes", that actually means "no". – Yannis Rizos♦ Mar 4 at 18:10
2  
@YannisRizos: Much like "Do you use the best tools money can buy?" (Yes ... wellllll ... within reason.) and "Do programmers have quiet working conditions?" (Ohhhh yes ... wellllll ... depends on your reference point for quiet, I guess.) – pdr Mar 4 at 18:17
@pdr Depends on whether you consider the sound of sirens coming in through open windows quiet. – Robert Harvey Mar 4 at 20:19
Also, "Yes, I do top-down design." ;) – Izkata Mar 4 at 21:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.