Spy vs Spy

Joe Ferris

We’ve recently been making use of an alternative to the traditional mock-and-stub pattern common in Ruby: the Test Spy.

What do you mean, spy

Test spies allow you to record method invocations for later verification. Basic usage goes something like this:

describe PostsController do
  it 'shows the given post on GET show' do
    post = stub('a post', to_param: '1')
    Post.stubs(find: post)

    get :show, id: post.to_param

    Post.should have_received(:find).with(post.to_param)
    should render_template(:show)
    should assign_to(:post).with(post)
  end
end

Compare that with the traditional expectation-based example:

describe PostsController do
  it 'shows the given post on GET show' do
    post = stub('a post', to_param: '1')
    Post.expects(:find).with(post.to_param).returns(post)

    get :show, id: post.to_param

    should render_template(:show)
    should assign_to(:post).with(post)
  end
end

This may seem like a subtle difference, but cleanly separating the test’s phases has real benefits.

Why would I let spies in my code

The traditional xunit-style test follows four phases:

  1. Setup: create the necessary preconditions for a test
  2. Exercise: run the code you’re trying to test
  3. Verification: make sure that you got the expected result
  4. Teardown: clean up after your test so that it doesn’t interact

When using fast-failing mocks, this process is turned on its head. During the setup, you preemptively “verify” that certain methods are called with certain parameters. If the method is called unexpectedly, it will fail immediately (during the exercise phase). If it doesn’t get called at all, it will fail after the test (during the teardown phase). Besides being counter-intuitive and hard to keep track of, this presents problems when attempting to use the “one testcase per fixture” pattern (common in Shoulda and RSpec suites), or worse, when trying to reuse stubs and behavioral assertions.

Sharing stubs in a context

Many developers like to test each independent requirement for a piece of behavior individually. Using a mock, that general setup goes like this:

describe PostsController, 'on GET show' do
  before(:each) do
    @post = stub('a post', to_param: '1')
    Post.expects(:find).with(@post.to_param).returns(@post)

    get :show, id: @post.to_param
  end

  it { should render_template(:show) }
  it { should assign_to(:post).with(@post) }
end

If the mock isn’t being used as expected, every example will fail with the same message. In addition, you can’t have an example that specifies “it should find the given user,” because that specification got swallowed by the before block. If you like to write your example descriptions up front, this can be pretty disappointing. Here’s the same example using a test spy:

describe PostsController, 'on GET show' do
  before(:each) do
    @post = stub('a post', to_param: '1')
    Post.stubs(find: @post)

    get :show, id: @post.to_param
  end

  it { should render_template(:show) }

  it 'should find and assign the given post' do
    Post.should have_received(:find).with(@post.to_param)
    should assign_to(:post).with(@post)
  end
end

In this case, the phases are cleanly separated, and you can specify and verify behavior naturally. The two independent requirements can be tested independently, and you’ll get the failures you’d want and expect.

Sharing stubs between tests

One other problem with mocks is that you can’t share the stub without sharing the built-in verification. That means that every time you reuse a mock, you’re retesting the same behavior. Here’s an example:

describe PostsController do
  it 'shows a published post on GET show' do
    post = stub('a post', to_param: '1')
    post.expects(:published? => true)
    Post.expects(:find).with(post.to_param).returns(post)
    get :show, id: post.to_param
    should render_template(:show)
    should assign_to(:post).with(post)
  end
end

describe '/posts/show' do
  it 'should display a post' do
    assigns[:post] = stub('a post', :published? => true, title: 'a title')
    render '/posts/show'
    template.should have_tag('h1', assigns[:post].title)
  end
end

Because the mock also sets an expectation, the stubbed post is difficult to reuse in other tests where the expected methods are unimportant. However, using a test spy, you can share stubs and only verify the parts that are important in a particular test:

module PostHelpers
  def stub_post(post_attrs = {})
    post_attrs = {
      to_param: '1',
      :published? => true,
      title: 'a title'
    }.update(post_attrs)
    stub('a post', post_attrs)
  end

  def stub_post!(post_attrs = {})
    returning stub_post do |post|
      Post.stubs(find: post)
    end
  end
end

describe PostsController do
  include PostHelpers

  it 'shows a published post on GET show' do
    post = stub_post!
    get :show, id: post.to_param
    Post.should have_received(:find).with(post.to_param)
    post.should have_received(:published?)
    should render_template(:show)
    should assign_to(:post).with(post)
  end
end

describe '/posts/show' do
  include PostHelpers

  it 'displays a post' do
    assigns[:post] = stub_post
    render '/posts/show'
    template.should have_tag('h1', assigns[:post].title)
  end
end

As your test suite grows, the ability to refactor repeated stubs into reusable creation methods will allow you to refactor your production code without correcting mocks and stubs in dozens of files. Also, because the verification phase is separate, common expectations can be pulled into reusable matchers and assertions:

# post.class.should have_received(:find).with(post.to_param)
should find(post)
# Post.should have_received(:new).with(post_attrs)
should build(Post, post_attrs)
# Post.should have_received(:new).with(post_attrs)
# post.should have_received(:save)
should build_and_save(Post, post_attrs)

How can I get these spies on my side

Test spies are supported by the RSpec Mocks and Bourne test double frameworks. We use RSpec Mocks as our primary default and Bourne when the existing test suite uses Mocha.