Writing an API client with test, staging, and production in mind

I was recently tasked with this story:

Background:
  Given I go to the sign up page
  And I fill in "Email" with "new@example.com"
  And I fill in "Password" with "password"

Scenario: Visitor signs up and does not want email
  When I press "Sign up"
  Then email service is not notified "new@example.com" signed up

Scenario: Visitor signs up and wants email
  When I check "Please email me"
  And I press "Sign up"
  Then email service is notified "new@example.com" signed up

The email service in question was internal, accessible via an HTTP API, and did not have a Ruby client library.

So, I had to write and test my own. The usual approach would involve using a HTTP stubbing library like Sham Rack or Artifice. I decided to try something different and see how it felt.

First, I needed an interface that I could test:

Then /^email service is notified "([^"]*)" signed up$/ do |email|
  EmailService.notifications.should include(email)
end

Then /^email service is not notified "([^"]*)" signed up$/ do |email|
  EmailService.notifications.should_not include(email)
end

I’m re-using a pattern (“store data in a simple array for easy state-based testing”) we’ve used before for Javascript integration testing Mixpanel.

So I have the start to an EmailService interface. I thought I wanted it invoked as part of an after_save callback on the User model, so here’s that spec:

describe User, 'who opts into email' do
  subject { build(:user, email_opt_in: true) }

  before do
    EmailService.stubs(:notify)
    subject.save
  end

  it 'notifies EmailService' do
    EmailService.should have_received(:notify).with(subject.email)
  end
end

describe User, 'who does not opt into email' do
  subject { build(:user, email_opt_in: false) }

  before do
    EmailService.stubs(:notify)
    subject.save
  end

  it 'does not notify EmailService' do
    EmailService.should have_received(:notify).never
  end
end

This is the stubbing and spying technique and uses RSpec, mocha, and bourne.

My thought process was that I needed an active verb, notify to invoke when the user is created, but that will store the invocation in a notifications array that the Cucumber step definition needs to check state (I don’t want to stub, spy, or mock in an integration test).

So, making the user spec pass isn’t bad:

require 'email_service/notifier'

class User < ActiveRecord::Base
  after_create do
    if email_opt_in?
      EmailService.notify(email)
    end
  end
end

Now, the EmailService can be spec’ed. We already had John Nunemaker’s HTTParty as a dependency in the app, and I only had to make one HTTP POST, so I knew I would be re-using HTTParty’s interface.

describe EmailService::Notifier, '#post' do
  subject { EmailService::Notifier }

  let(:email) { 'new-signup@example.com' }

  before do
    subject.stubs(:post)
    subject.new(email).post
  end

  it 'POSTs to email service with API_KEY and given email' do
    subject.should have_received(:post).with(
      subject::URL, query: { api_key: subject::PARAMS, email: email }
    )
  end
end

What I care about here is that during the one POST the app has to make, that the parameters are correct. This is close to hitting the live service as I’m willing to get without making an HTTP request.

I had an internal debate with myself while writing it over whether this is “stubbing the system under test” (considered bad practice). I decided “no” because despite the subject being stubbed, the system under test is actually the EmailService::Notifier#post method.

The stubbed and spied method is also mixed in from HTTParty, so I feel clean with this approach.

Making it pass:

require 'httparty'

module EmailService
  class << self
    attr_accessor :notifications
  end

  def self.notify(email, live = false)
    if Rails.env.production? || live
      Notifier.new(email).post
    else
      self.notifications << email
    end
  end

  class Notifier
    include HTTParty

    API_KEY = 12345
    URL = 'http://emailservice.example.com'

    def initialize(email)
      @url   = URL
      @email = email
    end

    def post
      self.class.post(@url, query: { api_key: API_KEY, email: @email })
    end
  end
end

Summary

• I’ve got an environment-specific conditional inside the true public interface for the service, EmailService.notify, which is used by the User model.
• In my integration test and in the development environment, all that happens is an array is populated, which makes it easy to confirm that it was notified correctly.
• I’ve got a unit test that makes sure the correct URL and API key are used.
• I’ve got a way to override EmailService.notify with the live flag so I can invoke it from the Rails console on production or staging when testing or debugging.

This almost takes longer to describe than to code but I’m curious what people think about this style of writing an API client with test, staging, and production environments in mind. How do you do things differently?

Written by Dan Croak.

Recipe: Google Calendar

A recipe for accessing Google Calendar data from Ruby.

Why?

Google Calendar is a great interface for a few people to share management of events. You might have a client that has this need. Neither you nor the client want to spend time or money duplicating an ‘admin interface’ for managing events. You want to focus development time on displaying those events in a custom way.

Ingredients

• icalendar gem
• open-uri from the standard library

Google Calendar

For a Google Calendar you own, navigate to the “Calendar Details” page:

Your calendar has an address and can be accessed as XML/RSS, iCalendar, or HTML.

A good example of the Google Calendar embed option is Betahouse’s calendar.

iCalendar

For this recipe, we’ll use the iCalendar format. I tried GET’ing the XML version and parsing it with Nokogiri but found the iCalendar version more manageable.

Copy the “ICAL” button’s value. Save it in a comment in your code for now. Also download it. We’ll use it for our spec:

describe Tour do
  before do
    ics   = File.join(File.dirname(__FILE__), 'local_copy.ics')
    @tour = Tour.new(ics)
  end

  it "should find upcoming gigs" do
    @tour.upcoming_gigs.all? { |gig| gig.dtstart.should > DateTime.now }
  end
end

Make it go:

require 'open-uri'
class Tour
  GOOGLE_ICS = "http://google.com/calendar/ical/you@gmail.com/public/basic.ics"

  attr_accessor :cal

  def initialize(ics = GOOGLE_ICS)
    self.cal = Icalendar.parse(open(ics).read).first
  rescue *HTTP_ERRORS => error
    HoptoadNotifier.notify(error)
  end

  def upcoming_gigs
    cal.events.select  { |event| event.dtstart > DateTime.now }.
               sort_by { |event| event.dtstart }
  end
end

The default behavior uses the remote Google ics file, but we set up a constructor to allow us to easily replace the file in our specs.

The rescue is optional. HTTP_ERRORS is an Array from Suspenders I use in these circumstances in combination with Hoptoad to be notified if the HTTP calls to a remote service stop working for some reason.

If you wanted to get really fancy, you could have a nightly cron job that downloads a new copy of the .ics file and runs the spec on CI.

Example web app

I happened to be doing this in order to style the calendar nicely in a web app. He’s my Sinatra route…

get '/tour' do
  erb :tour, :locals => { :gigs => Tour.new.upcoming_gigs }
end

.. and the relevant portion of the view:

<ul class="gigs">
  <% gigs.each do |gig| %>
    <li>
      <%= gig.dtstart.strftime("%B %d, %Y %I:%M %p") %>
      <a href="http://maps.google.com/maps?hl=en&q=<%= gig.location %>"><%= gig.summary %></a>
    </li>
  <% end %>
</ul>

Check out the live example.

Bon appétit!

Testing Rake’s Integration

So now you’re integration testing, because that’s what cool kids are doing these days. This tests the joints of your app, making sure that the model code is being called from the controller code which is being invoked by the user.

A good integration test, as we all know, is from the user’s perspective: “I click this”, “I fill in that”, etc.

But what of rake tasks? Those are integration points. If they go wrong you’re depending on cron telling you, or maybe it’s a task you run once a year by hand. Either way, you want to test it.

We already move all of the rake task into the model. This simplifies unit testing; rake tasks are now just one method call, and that method is isolation tested.

Here’s an integration test for a rake task that sends an email with all new users:

require 'test_helper'

class DailyEmailReportTest < ActionController::IntegrationTest
  # In order to keep updated, I should see an email
  # of all new users' profile in the past day

  should "send an email from the rake test for the specific task" do
    old_user = Factory(:user,
                       :created_at => 2.days.ago,
                       :description => "old description")
    new_user = Factory(:user,
                       :created_at => 1.hour.ago,
                       :description => "new description")
    ActionMailer::Base.deliveries.clear

    i_call_rake_task "notifications:daily:users"

    i_see_in_email "new description"
    i_do_not_see_in_email "old description"
  end
end

Two test helpers are defined to help out: #i_see_in_email and #i_do_not_see_in_email. In addition to those (they just look over ActionMailer::Base.deliveries), we have #i_call_rake_task. That’s where the magic happens.

The first step is to override Rake::Task#invoke_prerequisites to avoid reloading the environment:

require 'rake'

# Do not re-load the environment task
class Rake::Task
  def invoke_prerequisites(task_args, invocation_chain)
    @prerequisites.reject{|n| n == "environment"}.each do |n|
      prereq = application[n, @scope]
      prereq_args = task_args.new_scope(prereq.arg_names)
      prereq.invoke_with_call_chain(prereq_args, invocation_chain)
    end
  end
end

Then we define the helper:

class Test::Unit::TestCase
  def i_call_rake_task(task_name)
    # Make sure you're in the RAILS_ROOT
    oldpwd = Dir.pwd
    Dir.chdir(RAILS_ROOT)

    # Get an instance of rake
    rake_app = Rake.application
    rake_app.options.silent = true

    # Back to where you were
    Dir.chdir(oldpwd)

    rake_app.init
    rake_app.load_rakefile

    task = rake_app.tasks.detect {|t| t.name == task_name}
    assert_not_nil task, "No rake task defined: #{task_name}"
    task.reenable
    task.invoke
  end
end

With this in place, you can be more sure that rake tasks are integrated with your system. Wee!