Why Do Rubyists Test So Completely?

The Ruby community, according to some data I am making up, has the strongest test-driven development attitude. Not all of us TATFT; not all of us test most of the time; and not all of us test ever—but those who do test make up a larger proportion than they do in, say, the Java world.

But why? What is it about Ruby that drives us to attain 100% code coverage?

Cultural Pressure

We test. Kent Beck tests. Rails comes set up for you to write tests as part of your application. Are you such a skilled programmer, such a badass rebel, such a unique person, and such a loner that you wouldn’t dare test your Ruby code?

No, because your app would break and you know it. More importantly, everyone in the community knows it. If you publish it to Rubyforge and there are no tests, don’t expect us to use it. So you write tests, because we want you to.

(Thank you for doing that for us.)

It’s a Mindset

Test-driven development is cheating. You think upfront about the problem, the solutions, the problems with those solutions, and you document all this in code. It fails, with a pretty error message, so you make it pass.

But you already know this one.

nil

Any method, at any time, might produce nil. You never know when it might happen. Even instance variables might produce nil. Your code will keep going until this becomes an issue…and it will become an issue.

The reason nil appears so often, even when unexpected, is in part because of the implicit return at the end of methods, and in part because if in Ruby is different from if in C (it’s an expression that produces a value instead of a statement that just runs). This is different from what many people expect and this difference often goes ignored.

We’ve worked around this in places where we expect nil. Either we re-write the algorithm to #compact the nils out early, or we use #validate_presence_of and stop expecting nil, or we use the #try hack, or some other specific solution. But it lingers in the back of our mind that any method we’re calling could produce an absolutely useless value.

No static analysis

We have to write complete tests that verify the runtime code because there is no tool to automate this. In a world where even C can tell you when you are passing an integer where it expected a string—before even running your program—Ruby cannot do anything of the sort. Your program works until it stops working, and then you dig through Hoptoad for a minute, write a test for 30 minutes, fix the code (five minutes), and re-deploy. Live, while the client is waiting.

So we write the tests first so the client stays happy. Instead of just hacking away at code until it compiles, we write tests then hack away at code until they pass.

Work has been done to try to statically analyze Ruby. Our very own Jason Morrison did some work in 2006 for the Google Summer of Code to build a type inference engine, then tried to continue the project before abandoning the idea; according to him, the main problem is eval. Another problem he’s mentioned is the way classes are implemented, with the singleton classes and mixins and whatnot. Ping him for more information.

Refactoring Mutation

We write tests so we can refactor without care. What if we change the methods called, or the order of the callbacks; what will break?

This matters because code can depend on ordering. Variables can be mutated: set, unset, and changed from anywhere. We might print to the logger in one callback then follow-up that printing in another. We might depend on an instance variable not being nil in a method because of undocumented invariants. If we re-write a seemingly innocent method in a way that changes the order in which things are mutated, anything could go wrong.

So What?

So we test because Ruby provides no better alternative, and because testing is awesome. What if we tested only because we enjoyed it, not because we got anything technical out of it? What if our unit tests never failed except for when we first wrote them? What if regressions were caught by the language implementation instead of by custom code?

What changes would you make to Ruby to achieve the goal of never unit testing again?

Update: I was trying out Tumblr answers on this post without realizing that our theme does not support that. Please leave a Disqus comment instead!

Find A Third Way Home

The lesson for day two in my twenty five part series is “find a third way home”. As promised, this is a vague metaphor and I’ll now provide very little explanation for it.

Often when you’re working on a large project or even on a smaller specific problem you run into situations where everyone involved in the decision-making process feels pigeon-holed into making a “black and white” decision—and neither perceived choice is very appealing. It’s common to get frustrated, churn your wheels for days, re-iterate the pluses and minuses of the two choices (and believe me, the minuses always seem more bad than the pluses seem good), get bogged down in meetings where everyone defers making a decision, and so on.

It’s at times like these where one of the following is probably true. First, maybe you really do have something that’s a genuinely hard problem with no good solution. The world is full of these. Try breaking it down into many smaller problems and see if you can “feel good” about solving any of them. If not, repeat until you can. Secondly – and I believe this is very common – you may have fallen victim to setting up a set of alternatives which do not exhaust the set of ways to solve your problem. Maybe there’s a third way. Maybe there’s a fourth way. Maybe you can take the “good” half of each solution and create a safe solution that way.

As an example – I was in a chinese restaurant ordering dinner earlier tonight, and I couldn’t decide which soup to order. The egg drop has a bizarre yet satisfying texture to it – but generally tastes weird. The wonton is delicious partially because I don’t know quite what a wonton is – but is served in a broth unworthy of praise. As it turns out, you can get “wonton egg drop soup”.

The details can’t be generalized, but the pattern can. Look for a third way home.