Morendil comments on Coding Rationally - Test Driven Development - Less Wrong

25 Post author: DSimon 01 October 2010 03:20PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (82)

You are viewing a single comment's thread. Show more comments above.

Comment author: Morendil 02 October 2010 09:12:59AM *  1 point [-]

Let's use a more concrete example. Since I've recently worked in that domain, say we're implementing a template language like mustache.

We're starting from a blank slate, so our first test might be a test for the basic capability of variable substitution:

input: blah{{$foo}}blah
context: foo=blah
expectation: blahblahblah

Is this format clear? It's language-agnostic, so you could implement that test in Ruby or whatever.

We want to see this test fail. So we have to supply an implementation that is deliberately broken - a simple way to do that is to return an empty string, or perhaps return the exact same string that was passed as input - there are many ways to be broken.

At this point an experienced TDDer will notice this arbitrariness, think "we could have started with an even simpler test case", and write the following:

input: blahblah
context: foo=whatever
expectation: blahblah

We've narrowed it down to only one non-arbitrary way to make the test fail: return the empty string. And to make the test pass we'll return the original input.

See how this works? Because I'm not yet thinking about my implementation, but thinking about how my tests pin down the correct implementation I'm free to come up with non-confirmatory examples. My tests are drawing a box around something, but I'm not yet concerned with the contents of the box.

Now we can go back to the simple variable substitution. An important part of TDD is that one failing test does not allow you, yet, to write the fully general code of your algorithm. You're supposed to write the simplest possible code that causes all tests to pass, erring on the side of "simplistic" code.

So for instance you'd write a small method that did more or less this:

return "blahblahblah" if template.contains("{")
else return "blahblah"

Many of my colleagues make a long face when they realize this is what TDD entails. "Why would you write such a deliberately stupid implementation?" Precisely to keep thinking about better tests, and to hold off on thinking about the fully general implementation.

So now I may want to add the following:

input: blah{{$bar}}blah
context: foo=blah
expectation: blahblah

And maybe this:

input: blah{$foo}blah
context: foo=blah
expectation: blah{$foo}blah

Which are important non-confirmatory test cases. And I want to see these tests fail, because they reveal important differences between a sophisticated enough implementation and my "naive" first attempt.

I will also probably be thinking that even this "basic" capability is starting to look like a fairly complex bit, a complexity which wasn't obvious until I started thinking about all these non-confirmatory test cases. At this point if I were coding this for real I would start breaking down the problem, and maybe narrow the problem down to tokenizing the template:

input: blah
expectation: (type=text, value="blah")

and

input: blah{{$foo}}blah
expectation: (type=text, value="blah"), (type=var, value="foo"),(type=text, value="blah")

(Now I'm not testing the actual output of the program, of course, but an intermediate representation. That's OK, since those are unit tests: they're allowed to examine internals of the code.)

At this point the only line of code I have written is a deliberately simplistic implementation of my expansion algorithm, and I have already gotten a half-dozen important tests out of my thinking.

Comment author: DSimon 05 October 2010 01:51:29PM *  2 points [-]

This is a good explanation. I have one point of difference, though:

input: blah{{$foo}}blah

context: foo=blah

expectation: blahblahblah

return "blahblahblah" if template.contains("{") else return "blahblah"

This implementation has copy&pasted magic values from the test. I've usually thought of these kinds of intermediate implementations as being side tracks because AIUI they are necessarily weeded out right away by the refactor phase of each cycle.

So, my deliberately-stupid implementation might've been:

def substitute(input, context): return input.sub(/\${{.+?}}/, context.values.first)

Which is more complex than the one you suggested, but still I think the least complex one that makes the test pass without copy & paste.

Then as with your example, this would've led to tests to make sure the right substitution variable was being matched to the right key, in which more than one substitution variable is supplied, in which substitutions are made for variables that aren't in the context, and so on....

(By the way, how did you get that nice fixed-width font?)

Comment author: wnoise 05 October 2010 03:17:59PM *  1 point [-]

but still I think the least complex one that makes the test pass without copy & paste.

He didn't say "without copy and paste".

Come to think of it, "simplest" varies person to person. In one metric the "simplest that could work" would just be a huge switch statement mapping input for a given test to output for the same test...

(By the way, how did you get that nice fixed-width font?)

http://wiki.lesswrong.com/wiki/Comment_formatting

Enclose with backticks for inline code, and

 start with spaces for blocks.
Comment author: DSimon 06 October 2010 02:09:01PM *  0 points [-]

He didn't say "without copy and paste".

Just copying the expected value from the test into the body of the implementation will make the test go green, but it's completely un-DRY, so you'd have to rip it out and replace it with a non-c&p implementation during the necessary refactor phase anyways.

Wikipedia agrees with me on this, and they cite to "Test-Driven Development by Example" by Kent Beck, the original TDD guy.

So, TDD as I learned it discourages c&p from the test. However, Morendil, now you've got me interested in talking about the possible benefits of a c&p-permitted approach: for example, I can see how it might force the programmer to write more sophisticated tests. Though on the other hand, it might also force them to spend a lot more time on the tests but for only minor additional benefit.