Have you ever written tests so big, so comprehensive, that every time you change
something in your code, you have to change your tests accordingly? Have you
ever spent more than 30 minutes writing functional tests for a controller,
including individually submitting nils and invalid data to each action in turn?
How about, have you ever had to use class_eval to awkwardly construct a set of
functional tests that did similar things, because Rails says you can only do one
get/post call in a functional test? This is all nonsense. If you want to
stay sane as a tester, relax and write the tests that you need and not the
tests that your needs-100%-comprehensivity-at-all-times part of your brain
wants, you will find that you are a whole lot more rested, and in general
you’ll bleed a lot less.
This is not about eschewing testing, this is about saying No to Big Testing. Like Big Oil before it, Big Testing is gaining a lot of clout in the Ruby and Rails community. Plugins like Rcov and Heckle are emerging as ways to ensure your code is production-ready, and that in your flawed human state you have not erred and left out a crucial test. How will you compete with Google Docs & Spreadsheets if your web-based office application crashes when someone tries to take the square root of a negative number?
As long as your application has a certain critical mass of stability, stability has very little to do with whether your idea will take off or not. It’s all the design choices you’re making, and whether your visual design is appealing and professional. You know, the meat of your application. The skeleton underneath should be strong, it should work, but you don’t need to have so much monitoring equipment hooked up to it that every time you want to shift some bones around, you first have to untangle knots of electrical cords.
Here’s a code sample to break up this post and make it seem more visual. Don’t study it too hard, it doesn’t have any relevance:
def select_bird
self.stoked? ? @bald_eagle : @eagle
end
When I was first introduced to formal unit and functional testing in Rails, I understood it as a way to refactor with ease. If you wrote basic tests, that made sure all of your features worked as expected, you could confidently go into a total rewrite of how they worked under the hood. If your tests passed, you didn’t have to be nervous. This is why I love tests. Little tests. Simple tests. Tests that touch the outside only, to make sure that the application is what the tests ordered out of the catalog. If you’re changing tests when you change code without changing design, those tests are either poorly written, or ultimately meaningless. For testing at that level, internal or public QA will take up far less of your time and money.
The three different kinds of testing (unit, functional, and integration) necessarily attach themselves to your code at three levels of granularity, with unit testing being the most code-specific, followed by functional, and then integration testing (which barely cares about your code at all). I understand that it is impossible to write unit tests and functional tests that don’t straitjacket the specific implementation of your design to at least some significant degree. My answer to this, then, is to write far less of them. Spend less time checking values in the assigns hash, and spend more time on ensuring realistic uses of the system work properly.
Again:
# chapters seven and eight of the poignant guide are coming soon!
excited = "11"
Testing is an asymptotic curve. You can spend 1 hour on a set of basic tests that cover 80% of your code, or you can spend a large bag of hours on covering 99.9% of your application. Most of the time, it’s not worth it. Write a functional test for every action that ensures it works as expected, write any additional interesting, caveat-y functional tests that come to mind without you really thinking about it (but not if it takes too long!). Don’t worry about nils or negative numbers, just move on and write a few unit tests to check other interesting model interactions. Then spend an hour writing a few key integration tests that really take your application (or application-to-be, if you’re doing test-driven development) for a spin. Then go back to working on design mocks for your next feature, or spend time talking to the friends you’ve asked to try your application out.
If your application has gotten so big and well defined that you don’t expect there to be any significant refactoring, and the app has become so mission critical that near-100% stability is a requirement, and you’re ready to lock down the codebase into a stable, predictable machine, then you should get Rcov and Heckle and spend an indefinite amount of hours writing Big Tests. But if your application is at that point, that sounds pretty boring, and I would find/found another company.
