It’s Not TDD, It’s Design By Example

The word “test” in software is a very loaded term. The first time I came across a tester (as in, a person performing the quality assurance role) was in my first professional job. We were never taught about testing at college, as we were all there to learn to be “developers”.

It turns out there are lots of different kinds of testing. In a series of blog posts titled Agile Testing Directions, Brian Marick came up with a 2x2 matrix of test types, and describes the activities which happen in each corner of the matrix. If you haven’t read the blog posts, they can help to provide a vocabulary about the kinds of testing that most software teams undertake.

A lot has changed in the nearly 6 years since Brian wrote those posts. Test Driven Development is arguably the most well known and most popular part of the eXtreme Programming software development process. For several years, I’ve been giving talks and coaching teams on how to do Test Driven Development, and by far the most frustrating part of the process has been overcoming people’s expectations when you use the word “test”. For most developers, “testing” is something that’s done by “testers”, not something that’s done by developers. Even those developers who are savvy enough to be doing unit testing are still practicing an activity whose primary purpose is quality assurance.

What the name Test Driven Development has going against it is that it doesn’t properly express the purpose of TDD; namely, that it is a process designed to help you drive and iterate the design of your implementation at the unit level. The result of the design process is unit tests, but their primary purpose is not one of quality assurance; rather, it is an expression of the intended usage of the component under design. In this way, the “tests” that you are writing become the first client of your component, and come into being just before the component’s code is written. The rhythm in TDD is “write a test, watch it fail; write the production code, watch the test now pass; when prudent, refactor the code to increase clarity and remove duplication”.

The unit tests written by TDD have some quality assurance value as a secondary effect of the test, but that is not their primary goal. Their primary goal is to help you design the code, and to give you a safety net with which to refactor your code. Note that when I use the word refactor, I mean it in the classical sense: to change the internal implementation of a component without changing its externally observable behavior; that is, if you find yourself needing to change a unit test, you are not doing refactoring.

Frustrated by the misunderstanding of the purpose of TDD, my friends and I (Peter, Jim, Brian, Scott, and many others, all agile practitioners and coaches) decided to start calling it Test Driven Design. A small change, but it starts to focus on the fact that the process is about design. Unfortunately, that “test” word baggage is still in there, so our next iteration was then Example Driven Design. This worked well too, but “EDD” and “TDD” were still too close together and confusing.

The final iteration ended up being Design By Example (DbE). Now when I talk about TDD, I always call it Design By Example, and explain why we like this name better than TDD. Where TDD(esign) or EDD failed to get traction, people really seem to resonate to Design By Example.

~ ~ ~

As an aside, you’ll note that most of the unit testing frameworks on .NET have the word “test” in them a lot. NUnit started this with the [Test] attribute, which MbUnit adopted, and MSTest converted into [TestMethod]. When Jim and I set out to design xUnit.net, it originally had a [Test] attribute in it as well, which is how it was when we first talked about it publicly at the first ALT.NET event in Austin. Even before ALT.NET, we’d been lamenting the name [Test] for the attribute exactly because of this baggage with the word “test”. We wanted xUnit.net to be a framework that was first and foremost for TDD (erm, DbE) practitioners.

At the Austin event, we decided to rename [Test] to [Fact]. While some users have complained that this is an arbitrary change, we felt it was right because it removed the focus of the word “test” from the code that you were writing when doing DbE. It also lined up very well with the [Theory] feature we added to xUnit.net Extensions to support David Saff’s data theories. The essential difference is: a [Fact] is an expression of some condition which is invariant, whereas a [Theory] is an expression of a condition which is only necessarily true for the given set of data. As such, [Theory]s are driven with external data; if you provide invalid data, then the theory will potentially produce invalid results.

An example of a [Fact] is the condition which says, “if you have failed to properly initialize this object, when you call this method on it, it will throw InvalidOperationException”. There are no variations on this rule; it is always true. An example of a [Theory], using the classic FizzBuzz interview question, is the condition which says, “if you pass a number which is divisible by 3, but not by 5, the result should be ‘Fizz’.” For this theory, you pass values which meet the initial requirement; if you were to pass 4 or 15, though, the result would be a failure, but that doesn’t mean the theory is bad, it simply means the preconditions for the theory weren’t properly met.

Comments

[Jérémie]

It's just a naming issue but naming is so important !
I adopt Design by Example from now on.

People won't say, you have written unit tests, no need to test then...
'It's not unit tests, it's facts checking !'

[Darrel Miller]

This path seems very similar to that traveled by Dan North here[1]. Any reason why you see DbE as a better name than BDD?

[1] http://dannorth.net/introducing-bdd

[Brad Wilson]

@Darrel Miller,

I haven't found the "given/when/then" pattern to be the way I want to write my tests, personally.

[Antony Marcano]

I started writing a comment... and it turned into a full-blown blog-post.

In short...

-Adding another name into the mix is reducing cohesion... maybe we all need to do something about that...

-Renaming the [Test] attribute to [Fact] isn't as good as just calling it an [Example] - especially due to your preferred phrase "Design by Example"

-Otherwise, I like the sound of the name DbE... it has a nice ring to it.

Hope you find the post interesting:
http://www.testingreflections.com/node/view/7977

[Brad Wilson]

@Antony,

I liked the blog post. I will say that the reason I wouldn't want to rename [Fact] to [Example] is that both facts and theories are examples.

[Dave Astels]

Others have: Been there, done that. Check out the whole BDD thing... it's not just given/when/then. Have a look at RSpec and similar frameworks.

[softwareprojectobservations.blogspot.com]

Brad,


Great article.


I hope that this use of the word Design will help to explain to Manager, Project Managers and stakeholders exactly what we are doing as Developers.


So many times I hear people say “We have not got time to do TDD”. My response has always been “What, you don’t want me to do any design!”


Good luck with promoting the Design of TDD.


Steve


TDD Training Course.co.uk/

[Chuck vdL]

The hangup many devs and other folks have with the word 'test' is a bit amazing to me at times. So I in general view movements such as this (and especially BDD) as a good thing. But really I do think you're on the right track, as are North, Astels, and others who are doing BDD, which I think you are being way too quick to dismiss.

Given, When, Then might not be how YOU want to write the tests internally, but it makes a great way for someone else (like the customer, PO, QA, etc) to provide you concrete examples of exactly how they expect the product to behave. It allows them to use the language of their business (Domain Specific Lanaguage -DSL) Not only that but it gets YOU using the same language as well, which makes interfacing with the customer easier when it's time to get feedback, brainstorm on new features and stories etc.

That structure also makes it very clear what the initial conditions are, what the 'action' is, and what the expected outcome(s) are. It also makes it harder for them to create examples with multiple or coupled actions, etc which make for poor tests. It forces them to consider the initial conditions as well, something that's pretty important.

Then using a tool like RSpec and/or Cucumber, you can take their plain language and turn it into unit 'tests' (call them what you want, they are providing validation that something works as expected, that is fundimentally a 'test') Also when asked what 'tests' (or "facts checking" if you like) you are doing in the code, you can actually show them the BDD tests, which is something non-devs can understand.

But enough of hyping BDD.. I want to take one issue with your approach which is the idea of a 'theory'. I object to this. You (and any developer) should KNOW what your code does when presented with 'invalid' input. There should not be any fuzzy theories over what MIGHT happen if someone hands you 4 or 15 (or zero or -12 or a float), there should be known 'facts' for how it responds to such inputs. All input is evil, you need to be prepared for something that doesn't conform to the happy path.

[a b]

You're not designing by example, you are implementing by example.
I like DDI (Design Driven Implementation).