Unit and Functional Tests are as Useful as 100% Code Coverage

I’m going to try and keep this short: if you’re writing Rails applications and are not writing integration tests, you need to start. Today. At Thoughtbot, we write integration tests with Cucumber, although you could use Steak or any other tool.

Most developers know that achieving 100% C0 code coverage isn’t a worthwhile goal in the grand scheme of things. You can use coverage reports to help guide you in what you may want to write tests for, but the law of diminishing returns becomes a factor in attempting to achieve 100% coverage.

Without integration tests, an awesome test suite doesn’t really mean much. I could test each individual method on every model I write, cover every response for every controller, and even go so far as to testing all my views, but in the end, something is missing. I’m not testing how my application behaves. So, just as 100% coverage doesn’t really mean much, the same goes for an app that’s tested well but lacks integration tests.

Any amount of testing is better than nothing; I’m not advocating that everyone stops writing unit or functional tests.

If you’re writing a Rails app, you’re not writing just an API to your models. You’re providing users an experience, and that experience should be tested! Unit and functional tests do not have the scope to test your full application (they were never meant to.) That’s where acceptance tests enter the picture.

For example, let’s say I have these tests (ripped shamelessly from Clearance):

# user_test.rb
context "A user" do
  setup do
    @user     = Factory(:user)
    @password = @user.password
  end

  should "authenticate with good credentials" do
    assert ::User.authenticate(@user.email, @password)
    assert @user.authenticated?(@password)
  end

  should "not authenticate with bad credentials" do
    assert ! ::User.authenticate(@user.email, 'bad_password')
    assert ! @user.authenticated?('bad_password')
  end
end

# sessions_controller_test.rb
context "on POST to #create with good credentials" do
  setup do
    @user = Factory(:email_confirmed_user)
    @user.update_attribute(:remember_token, "old-token")
    post :create, :session => {
                    :email    => @user.email,
                    :password => @user.password }
  end

  should_set_the_flash_to /signed in/i
  should_redirect_to_url_after_create

  should_set_cookie("remember_token", "old-token", Clearance.configuration.cookie_expiration.call)

  should "not change the remember token" do
    assert_equal "old-token", @user.reload.remember_token
  end
end

By themselves, what do these test? They test that I have a controller that responds to the HTTP verb POST and will set the flash and sign me into the application, and that I can call #authenticate on a User with an email and password and get a boolean response.

Great. That doesn’t mean I can sign into the application from a browser. I could throw an assert_select or two in there, but where’s the overall behavior of going to the homepage, clicking “Login”, filling in the form, and hitting the “Sign In” button?

Let’s describe signing in with a Cucumber scenario (also shamelessly stolen from Clearance):

Scenario: User signs in successfully
  Given I am signed up and confirmed as "email@person.com/password"
  When I go to the sign in page
  And I sign in as "email@person.com/password"
  Then I should see "Signed in"
  And I should be signed in
  When I return next time
  Then I should be signed in

Between both of these, which do you think describes the ability for a user to sign in successfully?