Using Capybara to test JavaScript that makes HTTP requests

Testing an application that integrates with an HTTP service can be tricky:

• Making a connection to another server is slow.
• Your tests can become dependent on data that lives outside the test suite.
• Going offline means you can’t run the tests.
• The service going down causes continuous integration to fail.

It may seem like a world of pain, but you’re not going to let a few HTTP requests get between you and your TDD, are you?

Story Time

Let’s say you’re making an internal dashboard for your site, which allows you to view key health metrics. Among other things, you want to display the current status of the build, so that you know whether or not it’s safe to deploy. Your build runs on a third party service, so you need to query their API.

From the Top

You start with an acceptance test:

feature 'health dashboard' do
  scenario 'view health dashboard' do
    create_passing_build
    sign_in_as_admin

    view_health_dashboard

    page.should have_passing_build
  end

  def create_passing_build
    FakeContinuousIntegration.stub_build_message(passing_build_message)
  end

  def view_health_dashboard
    visit '/admin/health_dashboard'
  end

  def have_passing_build
    have_content(passing_build_message)
  end

  def passing_build_message
    'All 2,024 tests passed.'
  end
end

The test immediately fails because of your missing fake, and you TDD your way into this simple class:

class FakeContinousIntegration
  def self.stub_build_message(message)
    @@build_message = message
  end
end

Your testing loop leads to this controller action:

def show
  @latest_build_message = ContinousIntegration.latest_build_message
end

Details Emerge

At this point, it’s time to drop down into a unit test. After a few cycles, you end up with this test:

describe ContinousIntegration, '.latest_build_message' do
  it 'parses the build message from the CI server' do
    message = 'Great success'
    response = { 'message' => message }.to_json
    Net::HTTP.stubs(get: response)

    result = ContinousIntegration.latest_build_message

    Net::HTTP.should have_received(:get).with('buildserver.com', '/latest')
    result.should == message
  end
end

And the implementation emerges:

class ContinousIntegration
  HOST = 'buildserver.com'
  LATEST_BUILD_PATH = '/latest'

  def self.latest_build_message
    new(LATEST_BUILD_PATH).build_message
  end

  def initialize(path)
    @path = path
  end

  def build_message
    data['message']
  end

  private

  def data
    @data ||= JSON.parse(download_build)
  end

  def download_build
    Net::HTTP.get(HOST, @path)
  end
end

Connecting the Dots

With your unit test passing, you return to the integration test. At this point, you no longer receive any errors about missing constants or undefined methods. Instead, everything runs as you expect, but you’re getting a different build message: “All 126 tests passed.” Where did that come from? As the gears start turning, you realize that your test is fetching the actual build status.

There’s no reason to make an actual HTTP request in the test, so you reach for WebMock.

# in spec/support/fake_continuous_integration.rb
stub_request(:any, /buildserver.com/).to_rack(FakeContinuousIntegration)

Now any Net::HTTP requests to “buildserver.com” will route directly to your fake, rather than actually opening a request. All that’s left is to flesh out our fake a little more:

require 'sinatra/base'

class FakeContinousIntegration < Sinatra::Base
  def self.stub_build_message(message)
    @@build_message = message
  end

  get '/latest' do
    content_type :json
    { 'message' => @@build_message }.to_json
  end
end

Tests pass, page looks good. Time to ship.

Two Words: Java. Script.

It doesn’t take long before somebody decides that it’s not a good idea to query your build server in the middle of a request. Luckily, you realize that your build server comes fully equipped with a JSONP API, so you can offload that request to the browser:

// in app/assets/javascripts
function fetchBuildMessage(target) {
  $.ajax({
    url: 'http://buildserver.com/latest',
    dataType: 'jsonp',
    success: function(response) {
      $(target).text(response.message);
    }
  });
}

// in your .erb view
fetchBuildMessage('#buildMessage');

Of course, your fake doesn’t implement this JSON endpoint, so you have to fix that:

get '/latest' do
  callback = params[:callback]
  data = { 'message' => @@build_message }.to_json
  "#{callback}(#{data})"
end

You tag the scenario as javascript and let capybara do its magic, but even after fixing your fake, it’s regressed back to hitting the actual build server over HTTP. Testing this HTTP service was bad enough, and many developers shy away from testing their JavaScript, but the combination of the two is a formidable opponent. After coming this far, though, you’re ready to do what it takes.

What The World Needs Now Is Threads, More Threads

Tools like WebMock are great, but when testing JavaScript, it’s a seperate browser process that loads the page, and not your Ruby test process. That means that the request to your build server isn’t going through Net::HTTP; the requests are coming from Firefox or capybara-webkit, and those tools are gleefully unaware of your feeble attempts to reroute HTTP traffic. Fortunately, there are only two steps remaining towards the testing Holy Grail:

• The JavaScript is going to make an actual HTTP connection, so we need to have an actual HTTP server running somewhere with our fake.
• The JavaScript is talking to “buildserver.com,” which we don’t control, so we need to get it to use a configurable host.

We can use Capybara to solve the first issue. Instead of mounting the application using WebMock, we run it using Capybara::Server:

class FakeContinousIntegration < Sinatra::Base
  def self.boot
    instance = new
    Capybara::Server.new(instance).tap { |server| server.boot }
  end
  # ...
end

Next, we can put the CI host name in a constant. In most environments, this will be “buildserver.com”, but in the test environment, we can get the URL from the server we just spun up:

# config/environments/{development,staging,production}.rb
CI_HOST = 'buildserver.com'

# in spec/support/fake_continuous_integration.rb
server = FakeContinuousIntegration.boot
CI_HOST = [server.host, server.port].join(':')

Now we just need a parameter in our JavaScript function:

// in app/assets/javascripts
function fetchBuildMessage(host, target) {
  $.ajax({
    url: 'http://' + host + '/latest',
    dataType: 'jsonp',
    success: function(response) {
      $(target).text(response.message);
    }
  });
}

// in your .erb view
fetchBuildMessage('<%= CI_HOST %>', '#buildMessage');

Made it, ma! Top of the world!

Factory Girl callbacks

Factory Girl now has callbacks thanks to Nate Sutton.

There are three callbacks:

after_build
after_create
after_stub

When build(:user) is invoked, the after_build callback runs after building the user. Likewise for create and stub.

These come in handy in a number of common use cases.

Basic has many associations

Models:

class Article < ActiveRecord::Base
  has_many :comments
end

class Comment < ActiveRecord::Base
  belongs_to :article
end

Factories:

factory :article do
  body 'password'

  factory :article_with_comment do
    after_create do |article|
      create(:comment, article: article)
    end
  end
end

factory :comment do
  body 'Great article!'
end

Nice. Callbacks let us do this:

article = create(:article_with_comment)

Instead of this:

article = create(:article)
create(:comment, article: article)

Polymorphic relationships

The savings get larger when the object graph gets more complex:

Models:

class User < ActiveRecord::Base
  has_many :interests, as: :interested
  has_many :topics, through: :interests
end

class Interest < ActiveRecord::Base
  belongs_to :topic
  belongs_to :interested, polymorphic: true
end

Building block factories:

factory :user
  email
  password 'password'

  factory :email_confirmed_user do
    email_confirmed { true }
  end
end

factory :topic do
  name 'topic_name'
end

factory :interest
  topic
  interested factory: :user
end

factory :music_interest, class: 'Interest' do
  topic.association(:topic, name: 'Music')
end

factory :sports_interest, class: 'Interest' do
  topic.association(:topic, name: 'Sports')
end

And now the payoff:

factory :musical_user, parent: :email_confirmed_user do
  after_create { |user| create(:music_interest, interested: user)
end

factory :sporty_user, parent: :email_confirmed_user do
  after_create { |user| create(:sports_interest, interested: user) }
end

Again, factories let us do this:

user = create(:musical_user)

Instead of:

user = create(:email_confirmed_user)
create(:music_interest, interested: user)

More intention-revealing. More encapsulation, protecting us from change.

Working with fakes

Fakes are a good approach for testing objects that interface with web services such as geocoding and payment processing.

class User < ActiveRecord::Base
  acts_as_mappable
  before_validation :geocode_location, if: :location_changed?

  def geocode_location
    geo = Geokit::Geocoders::MultiGeocoder.geocode(location)
    self.lat, self.lng = geo.lat, geo.lng
  end
end

factory :user do
  factory :boston_user do
    location 'Boston, MA'

    after_build do |user|
      Geokit::Geocoders::FakeGeocoder.locations['Boston, MA'] = [0, 1]
    end
  end
end

Enjoy!

Install:

gem 'factory_girl'

Happy testing.

Written by Dan Croak.

Have you ever…faked it?

I’ll admit it - I’ve faked it. Sometimes, you just can’t wait for a service to finish and you just want to fake a satisfactory response. There are lots of techniques for doing this: stubs, mocks, spies, and fakes. A full fake object will require more up-front effort than a quick stub, but they can be more reusable, reliable, and fail-proof.

Keeping your tests local

Probably the first reason every developer encounters that drives them to test doubles of any kind is an external service. Writing tests that interact with a live server is bad for a number of reasons: it’s slow, it’s hard to setup your test, and your tests will likely interact with each other (or other developers’ tests). You’ll also eventually run into that frustrating situation where server downtime results in a development blackout.

Overriding methods

Ruby is extremely flexible with its class definitions, to the point that you’re allowed to reopen and append (or redefine) methods at any point. I’ve seen lots of projects where this is used to simply white out pieces of the application that developers don’t want running in tests:

# The production code
# app/models/event.rb

class Event < ActiveRecord::Base
  before_validation :geocode
  # ...
  private
  def geocode
    geo = GeoKit::Geocoders::MultiGeocoder.geocode(address)
    if geo.success
      self.lat, self.lng = geo.lat, geo.lng
    else
      self.errors.add(:address, 'Unable to identify your location.')
      false
    end
  end
end

# The override
# test/support/geocoding.rb

module GeoKit
  module Geocoders
    class MultiGeocoder < Geocoder
      def self.geocode(location)
        loc = GeoLoc.new
        loc.lat = 1
        loc.lng = 1
        loc.success = true
        loc
      end
    end
  end
end

That little snippet lets you hit the ground running. Your tests don’t need to hit an external server, and you can test that a latitude and longitude is assigned when the record is created. However, you can’t test the negative case, and if you want to write tests for geo-spatial search, you’re out of luck.

Using stubs

If you need different geocoded results in different tests, a straight-up override won’t do it for you. At this point, a developer might turn to stubs and mocks:

describe Event do
  it "should geocode a valid location" do
    loc = GeoLoc.new
    loc.lat = 100
    loc.lng = 200
    loc.success = true
    GeoKit::Geocoders::MultiGeocoder.
      stubs(:geocode).
      with('123 Happy Street').
      returns(loc)
    event = Factory.build(:event, :address => '123 Happy Street')
    event.save
    event.lat.should == loc.lat
    event.lng.should == loc.lng
  end

  it "should add an error message with an invalid location" do
    loc = GeoLoc.new
    loc.success = false
    GeoKit::Geocoders::MultiGeocoder.
      stubs(:geocode).
      with('123 Sad Lane').
      returns(loc)
    event = Factory.build(:event, :address => '123 Sad Lane')
    event.save
    event.should_not be_valid
  end
end

You’ll probably need to stub out geo locations several times throughout the test suite, so wrapping these up in helper methods helps:

describe Event do
  it "should geocode a valid location" do
    loc = build_geoloc(:lat => 100, :lng => 200, :success => true)
    stub_geoloc!('123 Happy Street' => loc)
    event = Factory.build(:event, :address => '123 Happy Street')
    event.save
    event.lat.should == loc.lat
    event.lng.should == loc.lng
  end

  it "should add an error message with an invalid location" do
    loc = build_geoloc(:success => false)
    stub_geoloc!('123 Sad Lane' => loc)
    event = Factory.build(:event, :address => '123 Sad Lane')
    event.save
    event.should_not be_valid
  end
end

However, this leads to a few new problems:

• Most of the time you create an Event in a test, you don’t care about geocoding, so you’ll probably still need to add an override.
• Writing stubs like this in Cucumber tests is tricky - you need to make sure your stubs are torn down, and refactoring your implementation will noisily cause scenarios to fail. This isn’t what you want from an integration test.
• You can’t test the stubs themselves, and these methods are difficult to reuse, especially between projects.

Swappable components

One of the best pieces of programming advice I’ve ever received is this: “separate the pieces that change from the pieces that don’t.” Logic that doesn’t directly apply to your domain, such as geocoding and credit card processing, are likely to change. Which provider will you use? Do you need failover support? Adding the code (and tests) to your models is likely to increase churn and noise in the essential models of your application. Extracting these other concerns to external components helps reduce this noise. Writing these components so that the implementation can be swapped out will take you even further.

Luckily, in the case of GeoKit, this has all been done for you. GeoKit supports several Geocoders, and the list of Geocoders that will be used can be configured per-environment. The list of Geocoders is configured via the global “provider_order” setting:

GeoKit::Geocoders::provider_order = [:fake]

GeoKit will look for a camelized constant nested under GeoKit::Geocoders for each provider specified, so you can write a fake geocoder like this:

module GeoKit
  module Geocoders
    class FakeGeocoder < Geocoder
      def self.geocode(location)
        # return a GeoLoc instance
      end
    end
  end
end

If you maintain a global registry of locations mapped to latitude and longitude, you can write steps like the following:

Given /^the following geolocations:$/ do |table|
  table.hashes.each do |hash|
    location = hash['Location']
    lat = hash['Lat'].to_f
    lng = hash['Lng'].to_f
    GeoKit::Geocoders::FakeGeocoder.locations[location] = [lat, lng]
  end
end

Because GeoKit allows you to swap out the geocoder implementation without changing your model code, there’s no test-specific code in your model, no overrides to look for, and no extra churn in your model if you decide to switch providers.

Keeping it fresh

One potential issue with the above faking strategy is that there are a number of globals involved: the GeoKit provider is specified globally, as is the registry of fake locations. This means that you’ll need a teardown phase to clear the registry. In this case, there’s little reason to swap out implementation at runtime or between tests. However, if you’re faking out something that changes live, these global won’t do.

When possible, I recommend accepting the external component as a parameter. If Event could take a GeoKit provider as a parameter, you could write tests like the following:

describe Event do
  it "should geocode a valid location" do
    fake_geocoder = FakeGeocoder.new('123 Happy Street' => [100, 200])
    event = Factory.build(:event, :address => '123 Happy Street',
                                  :geocoder => fake_geocoder)
    event.save
    event.lat.should == 100
    event.lng.should == 200
  end
end

There’s no teardown required. The FakeGeocoder is discarded after the test executes, so the mapped locations don’t live beyond the test.

Keeping it real

When faking out a component, make sure you adequately reproduce the actual expected behavior. When faking out payment gateways, you’ll need to cover a variety of error responses and so on. When faking out a geocoder backend, you may want to simulate timeout errors and other failures. Make sure you have enough test cases that your application will catch the edge cases that occur when using the real thing.

Apply liberally

Developers are forced to extract code into external components when it’s just not feasible to use the same code in the tests, as is the case when contacting external services. However, there are many pieces of behavior that can be extracted that don’t need to be: searching, authentication, and so on.

Although developers usually only use stubs and mocks when they need to, many developers eventually prefer the isolation, speed, and declaritive nature of such doubles. If you start experimenting with fakes in place of mocks for external services, you may find that there are pieces of your code that you could swap out just to keep the churn down in your models, or to keep your tests focused on what they’re testing.

How about you? Have you ever…faked it?