Faster tests: sign in through the back door

One way to make tests faster is to avoid loading and submitting the sign in form during the setup phase.

This back door inserts Rack middleware into a Rails app that uses Clearance:

# config/environments/test.rb
class ClearanceBackDoor
  def initialize(app)
    @app = app
  end

  def call(env)
    @env = env
    sign_in_through_the_back_door
    @app.call(@env)
  end

  private

  def sign_in_through_the_back_door
    if user_id = params['as']
      user = User.find(user_id)
      @env[:clearance].sign_in(user)
    end
  end

  def params
    Rack::Utils.parse_query(@env['QUERY_STRING'])
  end
end

MyRailsApp::Application.configure do
  # ...
  config.middleware.use ClearanceBackDoor
  # ...
end

Then, include a user in an as parameter in integration tests:

visit root_path(as: user)

It works for any URL:

visit new_feedback_path(as: giver)

This is similar to Mislav’s approach except the Rack middleware works with Rails routing constraints.

On one project using this technique, the total test suite time was reduced 23%.

Written by Dan Croak.

January Online Workshops

We have two online workshops scheduled for January, 2013. If you’ve always wanted to take one of our workshops, but couldn’t travel to Boston or SF, these online versions will give you the same great education in a format custom-tailored to the online experience.

Design for Developers. This workshop is for developers who want to better understand the language and practice of visual design. This workshop starts January 4th, 2013.

Test-Driven Rails. By the end of the workshop, you will know how to do Test-Driven Development using RSpec and Capybara. This workshop starts January 11th, 2013.

Each of the workshops lasts 4 weeks and is a mix of pre-recorded lessons with extended periods of hands-on practice, one-on-one support, and group online office hours. Signing up today gets you the first video lesson while you wait for the workshop to start.

After the course ends, you’ll still have access to the course videos, instructor, and thoughtbot team who will answer any ongoing questions you have about the topic. You’ll also have access to any updates we make to the course in the future.

And as with everything at learn.thoughtbot.com, if you’re not happy, just let us know within 30 days and we’ll refund your money. It’s as simple as that.

I hope we’ll see you in one of the courses!

Using Capybara to test JavaScript that makes HTTP requests

Testing an application that integrates with an HTTP service can be tricky:

• Making a connection to another server is slow.
• Your tests can become dependent on data that lives outside the test suite.
• Going offline means you can’t run the tests.
• The service going down causes continuous integration to fail.

It may seem like a world of pain, but you’re not going to let a few HTTP requests get between you and your TDD, are you?

Story Time

Let’s say you’re making an internal dashboard for your site, which allows you to view key health metrics. Among other things, you want to display the current status of the build, so that you know whether or not it’s safe to deploy. Your build runs on a third party service, so you need to query their API.

From the Top

You start with an acceptance test:

feature 'health dashboard' do
  scenario 'view health dashboard' do
    create_passing_build
    sign_in_as_admin

    view_health_dashboard

    page.should have_passing_build
  end

  def create_passing_build
    FakeContinuousIntegration.stub_build_message(passing_build_message)
  end

  def view_health_dashboard
    visit '/admin/health_dashboard'
  end

  def have_passing_build
    have_content(passing_build_message)
  end

  def passing_build_message
    'All 2,024 tests passed.'
  end
end

The test immediately fails because of your missing fake, and you TDD your way into this simple class:

class FakeContinousIntegration
  def self.stub_build_message(message)
    @@build_message = message
  end
end

Your testing loop leads to this controller action:

def show
  @latest_build_message = ContinousIntegration.latest_build_message
end

Details Emerge

At this point, it’s time to drop down into a unit test. After a few cycles, you end up with this test:

describe ContinousIntegration, '.latest_build_message' do
  it 'parses the build message from the CI server' do
    message = 'Great success'
    response = { 'message' => message }.to_json
    Net::HTTP.stubs(get: response)

    result = ContinousIntegration.latest_build_message

    Net::HTTP.should have_received(:get).with('buildserver.com', '/latest')
    result.should == message
  end
end

And the implementation emerges:

class ContinousIntegration
  HOST = 'buildserver.com'
  LATEST_BUILD_PATH = '/latest'

  def self.latest_build_message
    new(LATEST_BUILD_PATH).build_message
  end

  def initialize(path)
    @path = path
  end

  def build_message
    data['message']
  end

  private

  def data
    @data ||= JSON.parse(download_build)
  end

  def download_build
    Net::HTTP.get(HOST, @path)
  end
end

Connecting the Dots

With your unit test passing, you return to the integration test. At this point, you no longer receive any errors about missing constants or undefined methods. Instead, everything runs as you expect, but you’re getting a different build message: “All 126 tests passed.” Where did that come from? As the gears start turning, you realize that your test is fetching the actual build status.

There’s no reason to make an actual HTTP request in the test, so you reach for WebMock.

# in spec/support/fake_continuous_integration.rb
stub_request(:any, /buildserver.com/).to_rack(FakeContinuousIntegration)

Now any Net::HTTP requests to “buildserver.com” will route directly to your fake, rather than actually opening a request. All that’s left is to flesh out our fake a little more:

require 'sinatra/base'

class FakeContinousIntegration < Sinatra::Base
  def self.stub_build_message(message)
    @@build_message = message
  end

  get '/latest' do
    content_type :json
    { 'message' => @@build_message }.to_json
  end
end

Tests pass, page looks good. Time to ship.

Two Words: Java. Script.

It doesn’t take long before somebody decides that it’s not a good idea to query your build server in the middle of a request. Luckily, you realize that your build server comes fully equipped with a JSONP API, so you can offload that request to the browser:

// in app/assets/javascripts
function fetchBuildMessage(target) {
  $.ajax({
    url: 'http://buildserver.com/latest',
    dataType: 'jsonp',
    success: function(response) {
      $(target).text(response.message);
    }
  });
}

// in your .erb view
fetchBuildMessage('#buildMessage');

Of course, your fake doesn’t implement this JSON endpoint, so you have to fix that:

get '/latest' do
  callback = params[:callback]
  data = { 'message' => @@build_message }.to_json
  "#{callback}(#{data})"
end

You tag the scenario as javascript and let capybara do its magic, but even after fixing your fake, it’s regressed back to hitting the actual build server over HTTP. Testing this HTTP service was bad enough, and many developers shy away from testing their JavaScript, but the combination of the two is a formidable opponent. After coming this far, though, you’re ready to do what it takes.

What The World Needs Now Is Threads, More Threads

Tools like WebMock are great, but when testing JavaScript, it’s a seperate browser process that loads the page, and not your Ruby test process. That means that the request to your build server isn’t going through Net::HTTP; the requests are coming from Firefox or capybara-webkit, and those tools are gleefully unaware of your feeble attempts to reroute HTTP traffic. Fortunately, there are only two steps remaining towards the testing Holy Grail:

• The JavaScript is going to make an actual HTTP connection, so we need to have an actual HTTP server running somewhere with our fake.
• The JavaScript is talking to “buildserver.com,” which we don’t control, so we need to get it to use a configurable host.

We can use Capybara to solve the first issue. Instead of mounting the application using WebMock, we run it using Capybara::Server:

class FakeContinousIntegration < Sinatra::Base
  def self.boot
    instance = new
    Capybara::Server.new(instance).tap { |server| server.boot }
  end
  # ...
end

Next, we can put the CI host name in a constant. In most environments, this will be “buildserver.com”, but in the test environment, we can get the URL from the server we just spun up:

# config/environments/{development,staging,production}.rb
CI_HOST = 'buildserver.com'

# in spec/support/fake_continuous_integration.rb
server = FakeContinuousIntegration.boot
CI_HOST = [server.host, server.port].join(':')

Now we just need a parameter in our JavaScript function:

// in app/assets/javascripts
function fetchBuildMessage(host, target) {
  $.ajax({
    url: 'http://' + host + '/latest',
    dataType: 'jsonp',
    success: function(response) {
      $(target).text(response.message);
    }
  });
}

// in your .erb view
fetchBuildMessage('<%= CI_HOST %>', '#buildMessage');

Made it, ma! Top of the world!

Let’s not

RSpec is an excellent test framework with a large community and an active team of maintainers. It sports a powerful DSL that can make testing certain things much easier and more pleasant.

However, there are a few features of RSpec’s DSL that are frequently overused, leading to an increase in test maintenance and a decrease in test readability.

Let’s look at an example from factory_girl’s test suite and see how we can improve it by favoring plain Ruby methods over DSL constructs:

describe FactoryGirl::EvaluatorClassDefiner do
  let(:simple_attribute) {
    stub("simple attribute", name: :simple, to_proc: -> { 1 })
  }
  let(:relative_attribute) {
    stub("relative attribute", name: :relative, to_proc: -> { simple + 1 })
  }
  let(:attribute_that_raises_a_second_time) {
    stub(
      "attribute that would raise without a cache",
      name: :raises_without_proper_cache,
      to_proc: -> { raise "failed" if @run; @run = true; nil }
    )
  }
  let(:attributes) {
    [
      simple_attribute,
      relative_attribute,
      attribute_that_raises_a_second_time
    ]
  }
  let(:class_definer) {
    FactoryGirl::EvaluatorClassDefiner.new(
      attributes,
      FactoryGirl::Evaluator
    )
  }
  let(:evaluator) {
    class_definer.evaluator_class.new(
      stub("build strategy", add_observer: true)
    )
  }

  it "adds each attribute to the evaluator" do
    evaluator.simple.should eq 1
  end

  it "evaluates the block in the context of the evaluator" do
    evaluator.relative.should eq 2
  end

  # More tests
end

A General Fixture is declared at the top. The fixture is then reused and augmented by each test to create the necessary setup. The examples (the it blocks) don’t declare any test setup; instead, they reference relevant portions of the existing fixture.

This approach causes a number of issues:

• It obscures each test by introducing a Mystery Guest.
• It causes Fragile Tests by creating a complicated fixture that is difficult to maintain.
• It causes Slow Tests by creating more data than is necessary in each test.

Will our mystery guest please leave

Addressing the Mystery Guest issue solves the largest concern: readability. A mystery guest causes obscure tests. Gerard Meszaros defines the Mystery Guest in his xUnit Patterns:

The test reader is not able to see the cause and effect between fixture and
verification logic because part of it is done outside the Test Method.

Here are some examples from this test suite:

it "adds each attribute to the evaluator" do
  evaluator.simple.should eq 1
end

it "evaluates the block in the context of the evaluator" do
  evaluator.relative.should eq 2
end

Without context, the reader has no idea what’s happening in this test, and the example description can’t really help. By parsing out the large fixture above, the reader can determine what’s going on, but correlating the fixture and test is slow and error-prone.

Let’s start by in-lining the fixture for this example:

it "evaluates the block in the context of the evaluator" do
  simple_attribute =
    stub("simple attribute",   name: :simple, to_proc: -> { 1 })
  relative_attribute =
    stub("relative attribute", name: :relative, to_proc: -> { simple + 1 })
  attribute_that_raises_a_second_time =
    stub("attribute that would raise without a cache",
         name: :raises_without_proper_cache,
         to_proc: -> { raise "failed" if @run; @run = true; nil })

  attributes = [
    simple_attribute,
    relative_attribute,
    attribute_that_raises_a_second_time
  ]
  class_definer = FactoryGirl::EvaluatorClassDefiner.new(
    attributes,
    FactoryGirl::Evaluator
  )
  evaluator = class_definer.evaluator_class.new(
    stub(
      "build strategy",
      add_observer: true
    )
  )

  evaluator.simple.should eq 1
end

The test continues to pass. Looking through the expected result, we can see that some data isn’t actually used in this scenario. Let’s remove it:

it "adds each attribute to the evaluator" do
  simple_attribute =
    stub("simple attribute",   name: :simple, to_proc: -> { 1 })

  attributes =
    [simple_attribute]
  class_definer = FactoryGirl::EvaluatorClassDefiner.new(
    attributes,
    FactoryGirl::Evaluator
  )
  evaluator = class_definer.evaluator_class.new(
    stub("build strategy", add_observer: true)
  )

  evaluator.simple.should eq 1
end

Now let’s in-line the fixture and remove unrelated data for the second example:

it "evaluates the block in the context of the evaluator" do
  simple_attribute =
    stub("simple attribute",   name: :simple, to_proc: -> { 1 })
  relative_attribute =
    stub("relative attribute", name: :relative, to_proc: -> { simple + 1 })

  attributes =
    [simple_attribute, relative_attribute]
  class_definer = FactoryGirl::EvaluatorClassDefiner.new(
    attributes,
    FactoryGirl::Evaluator
  )
  evaluator = class_definer.evaluator_class.new(
    stub("build strategy", add_observer: true)
  )

  evaluator.relative.should eq 2
end

Now that we’ve in-lined these two fixtures, there’s obviously a lot of duplicated setup logic. Let’s extract all that to a few factory methods:

it "adds each attribute to the evaluator" do
  attribute = stub_attribute(:attribute) { 1 }
  evaluator = define_evaluator(attributes: [attribute])

  evaluator.attribute.should eq 1
end

it "evaluates the block in the context of the evaluator" do
  dependency_attribute = stub(
    "dependency",
    name: :dependency, to_proc: -> { 1 }
  )
  dependency_attribute = stub_attribute(:dependency) { 1 }
  attribute = stub_attribute(:attribute) { dependency + 1 }
  evaluator = define_evaluator(
    attributes: [dependency_attribute, attribute]
  )

  evaluator.attribute.should eq 2
end

def define_evaluator(arguments = {})
  evaluator_class = define_evaluator_class(arguments)
  evaluator_class.new(FactoryGirl::Strategy::Null)
end

def define_evaluator_class(arguments = {})
  evaluator_class_definer = FactoryGirl::EvaluatorClassDefiner.new(
    arguments[:attributes] || [],
    arguments[:parent_class] || FactoryGirl::Evaluator
  )
  evaluator_class_definer.evaluator_class
end

def stub_attribute(name = :attribute, &value)
  value ||= -> {}
  stub(name.to_s, name: name.to_sym, to_proc: value)
end

Once we convert the remaining examples, we can delete the let statements that created the general fixture.

And the winner is…everyone

These converted examples are greatly improved:

• They’re easier to read, because all the actors referenced from the verification step are declared in the setup step within the it block.
• They’re less brittle, because each example only specifies the information it needs.
• They’re faster, because each example is running with a smaller data set.

It turns out that removing the Mystery Guests also solved our other complaints with these tests.

An added benefit is that the factory methods we created are easier to reuse throughout the test suite, whereas let statements are too specific to the examples for each example group. In time, this approach will make the entire test suite easier to maintain.

Until you need to break out the big guns like shared examples, avoid DSL constructs like subject, let, its, and before. Stick to your old friends: variables, methods, and classes.

Next Steps & Related Reading

Detect emerging problems in your codebase with Ruby Science. We’ll deliver solutions for fixing them, and demonstrate techniques for building a Ruby on Rails application that will be fun to work on for years to come.

Grab a free sample of Ruby Science today!