Unit and Functional Tests are as Useful as 100% Code Coverage

Note: code samples updated October 25, 2012

I’m going to try and keep this short: if you’re writing Rails applications and are not writing integration tests, you need to start. Today. At Thoughtbot, we write integration tests with RSpec and Capybara although you could use Cucumber, or any other tool.

Most developers know that achieving 100% C0 code coverage isn’t a worthwhile goal in the grand scheme of things. You can use coverage reports to help guide you in what you may want to write tests for, but the law of diminishing returns becomes a factor in attempting to achieve 100% coverage.

Without integration tests, an awesome test suite doesn’t really mean much. I could test each individual method on every model I write, cover every response for every controller, and even go so far as to testing all my views, but in the end, something is missing. I’m not testing how my application behaves. So, just as 100% coverage doesn’t really mean much, the same goes for an app that’s tested well but lacks integration tests.

Any amount of testing is better than nothing; I’m not advocating that everyone stops writing unit or functional tests.

If you’re writing a Rails app, you’re not writing just an API to your models. You’re providing users an experience, and that experience should be tested! Unit and functional tests do not have the scope to test your full application (they were never meant to.) That’s where acceptance tests enter the picture.

For example, let’s say I have these controller tests from the Clearance gem:

# spec/controllers/sessions_controller_spec.rb
describe Clearance::SessionsController do
  describe 'on GET to /sessions/new' do
    before { get :new }

    it { should respond_with(:success) }
    it { should render_template(:new) }
    it { should_not set_the_flash }
  end

  describe 'on POST to #create with good credentials' do
    before do
      @user = create(:user)
      @user.update_attribute :remember_token, 'old-token'
      post :create, :session => { :email => @user.email, :password => @user.password }
    end

    it { should redirect_to_url_after_create }

    it 'sets the user in the clearance session' do
      controller.current_user.should == @user
    end

    it 'should not change the remember token' do
      @user.reload.remember_token.should == 'old-token'
    end
  end
  # ...
end

By themselves, what do these test? They test that I have a controller that responds to the HTTP verb POST and will set the flash and sign me into the application, and that I can call #authenticate on a User with an email and password and get a boolean response.

Great. That doesn’t mean I can sign into the application from a browser. I could throw an assert_select or two in there, but where’s the overall behavior of going to the homepage, clicking “Login”, filling in the form, and hitting the “Sign In” button?

Let’s use Capybara to drive a headless web browser in an extended example from:

# spec/integration/visitor_signs_in_spec.rb
feature 'Visitor signs in' do
  scenario 'with valid email and password' do
    create_user 'user@example.com', 'password'
    sign_in_with 'user@example.com', 'password'

    user_should_be_signed_in
  end

  scenario 'tries with invalid password' do
    create_user 'user@example.com', 'password'
    sign_in_with 'user@example.com', 'wrong_password'

    page_should_display_sign_in_error
    user_should_be_signed_out
  end

  private

  def create_user(email, password)
    FactoryGirl.create(:user, :email => email, :password => password)
  end

  def sign_in_with(email, password)
    visit sign_in_path
    fill_in 'session_email', :with => email
    fill_in 'session_password', :with => password
    click_button "Sign in"
  end

  def user_should_be_signed_in
    visit root_path
    page.should have_content('Sign out')
  end

  def user_should_be_signed_out
    page.should have_content('Sign in')
  end

  def page_should_display_sign_in_error
    page.should have_css('div.error', 'Incorrect email or password')
  end
end

Between both of these, which do you think describes the ability for a user to sign in successfully?

Related Reading

End-to-end testing with RSpec integration tests and Capybara

The Quest Continues: Introducing capybara-webkit