[Here's a repost from 2003 Feb 1, when I first reported to this group about
the Mueller Hagner paper "Experiment about Test-first programming". Others
pointed the test should have run more than 1 iteration; that's the benefit.]
Here's a clinical experiment (from 2001, I think) about test-first. It was
before the TDD book came out, so of course someone tried to use it to refute
the TDD book.
It would be nice if the experiment address TDD instead of Test-First. TDD is
TF + Simple Design plus Refactoring. But the paper dismisses Simple Design
as "another XP practice" and hence rejects it to maintain purity.
Next, the "without test-first" team runs without unit tests period. But both
teams must pass an acceptance test. This is not incremental (a real team
would >start< with the acceptance test). but of course a "real team"
wouldn't have been experimental.
The paper concludes that the test-first team was less reliable (with a
significance p = 0.03) because they failed the acceptance test more often.
The test-first group implemented, and then passed the acceptance tests a
The test-first group implemented way faster; almost only half the time.
The no-first group passed the acceptance tests faster.
Of course a troll can decide that 1 data point out of 3 "against" TDD means
we should throw away TDD. The experiment itself is very interesting, and one
can easily surmise how the slower team spent more time thinking about what
the AT might do instead of writing their own tests. And, of course, the
test-first group came out with much more tests at the end ;-)
"The test-first group had less errors when reusing an existing method more
than once." That's >the point< here. The tested method was already re-used
once, in the test.
The paper leaves open the ideal that this simple effect compounds itself
over and over again, as the project gets larger, to constrain the design
space smaller and smaller. So a bigger test should show this contraction at
-- MCCD - Microsoft Certified Co-Dependent --