Turnip: a tasty Cucumber alternative?

We’ve used Cucumber heavily and successfully on client work, internal projects, and open source. We also love RSpec, so when we heard that Turnip would give the ability to run Gherkin based integration tests in our RSpec suite it was a no-brainer for us to try it out on a project.

Highlights of Turnip

• Integrates directly into your RSpec test suite
• Features and Step definitions live in the spec directory
• No need to maintain two configuration files
• Uses Ruby style symbols instead of regular expressions in step definitions
• No need to write Given/When/Then in the step definitions file
• Speed boost when running unit tests and integration tests together

We use Cucumber integration tests in thoughtbot’s clearance gem. In the example below I remove Cucumber and replace it with Turnip.

Steps used to change from Cucumber to Turnip

• Set up your features directory inside the spec directory
• Remove the Given/When/Then’s from the step definitions
• Replace regular expressions in the step definitions with Ruby style symbols

Cucumber and Turnip use the same Gherkin syntax:

 Scenario: Visitor signs up with valid data
   When I sign up with "email@example.com" and "password"
   Then I should be signed in 

Change the step definitions from Cucumber to Turnip style:

 # Cucumber step definition
 When /^I sign up (?:with|as) "(.*)" and "(.*)"$/ do |email, password| 
   visit sign_up_path 
   page.should have_css("input[type='email']") 
   fill_in "Email", :with => email 
   fill_in "Password", :with => password 
   click_button "Sign up" 
 end 

 # Turnip step definition
 step "I sign in with/as :email and :password" do |email, password|
   visit sign_in_path 
   page.should have_css("input[type='email']")
   fill_in "Email", :with => email 
   fill_in "Password", :with => password 
   click_button "Sign in"
 end 

Run the test suite

An advantage having everything running through RSpec is we get an immediate boost in speed when running the whole test suite. With Cucumber running Rake will run the RSpec and Cucumber tests (the Rails environment will be loaded twice). With Turnip all tests run through directly through RSpec (the Rails environment is only loaded once).

We save around 12 seconds when running the entire suite:

 # Rake running RSpec and Cucumber
 ~/Development/clearance_cucumber(master) $ time rake
 /Users/training/.rvm/rubies/ruby-1.9.2-p290/bin/ruby -S rspec ./spec/controllers/pages_controller_spec.rb ./spec/helpers/application_helper_spec.rb
 .

 Finished in 0.10696 seconds
 1 example, 0 failures
 /Users/training/.rvm/rubies/ruby-1.9.2-p290/bin/ruby -S bundle exec cucumber --profile default
 Using the default profile...
 .................................................

 13 scenarios (13 passed)
 49 steps (49 passed)
 0m1.729s
 rake 19.97s user 2.71s system 98% cpu 22.960 total

 # RSpec with Turnip
 ~/Development/clearance_turnip(master?) $ time rspec
 ..............

 Finished in 1.84 seconds
 14 examples, 0 failures, 0 pending
 rspec 8.85s user 1.06s system 95% cpu 10.362 total

Takeaways

• Great light-weight solution for anyone already using RSpec
• Step definitions easier to read than their Cucumber counterparts
• Low barrier to entry for developers new to integration testing

Javascript integration testing example: installing and using Mixpanel

We’ve been on a quest for years to make sure our integration tests covered the Javascript components of the app. We noted in November that we felt the community had reached an important plateau with a toolset of Cucumber, Capybara, and Akephalos.

While still frequently painful, our default mentality on new projects is “we will be able to test the Javascript components of this app in our usual integration tests.”

Here’s an example of this style of testing: installing Mixpanel in Copycopter to track visits, user sign ups, user activity during the free trial, and subscriptions.

features/mixpanel.feature:

Background:
  Given the following plan exists:
    | id | name       | price | trial |
    | 16 | Supersonic | 5     | false |
    | 5  | Trial      | 0     | true  |
  And the following limits exist:
    | plan             | name     | value |
    | name: Supersonic | users    | 13    |
    | name: Supersonic | projects | 14    |

@javascript
Scenario: Track visitor learning about Copycopter
  When I go to the homepage
  Then mixpanel should track the "visited-home" event
  When I follow "Take a tour"
  Then mixpanel should track the "clicked-tour" event
  When I follow "Next, view plans and pricing"
  Then mixpanel should track the "clicked-next-to-plans-and-pricing" event

@javascript
Scenario: Track visitor signing up for free trial
  When I go to the homepage
  And I follow "Plans and Pricing"
  Then mixpanel should track the "clicked-plans-and-pricing" event
  When I follow "Choose free trial"
  Then mixpanel should track the "viewed-plan" event with the properties:
    | plan_id | 5 |

We use the @javascript tag, which will use Akephalos in our setup.

We explicitly set the id’s of the ActiveRecord objects so we can check that Mixpanel receives the right plan id’s using their properties feature.

features/step_definitions/mixpanel_steps.rb:

Then %r{^mixpanel should track the "(.*)" event$} do |event_name|
  mpq = JSON.parse(evaluate_script(%{JSON.stringify(mpq);}))
  mpq.should include(["track", event_name])
end

Then %r{^mixpanel should track the "(.*)" event with the properties:$} do |event_name, table|
  mpq        = JSON.parse(evaluate_script(%{JSON.stringify(mpq);}))
  properties = table.transpose.hashes.first
  mpq.should include(["track", event_name, properties])
end

This is a little funky. We’re using JSON.stringify via json2.js and then Ruby’s JSON.parse to convert Mixpanel’s mpq Javascript object into its Ruby equivalent in order to invoke expectations on it.

Therefore, we need to include json2.js in our app:

curl https://github.com/douglascrockford/JSON-js/raw/master/json2.js > public/javascripts/json2.js

app/views/shared/_javascript.html.erb:

<% if Rails.env.test? %>
  <%= javascript_include_tag "json2" %>
<% end %>

That smells like a hack, but whatever…

Also in that partial, the actual setup for Mixpanel:

<script>
  var mpq = [];
  <% if Rails.env.staging? || Rails.env.production? -%>
    mpq.push(["init", "<%= MIXPANEL_TOKEN %>"]);
    (function() {
      var mp = document.createElement("script"); mp.type = "text/javascript"; mp.async = true;
      mp.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') + "//api.mixpanel.com/site_media/js/api/mixpanel.js";
      var s = document.getElementsByTagName("script")[0]; s.parentNode.insertBefore(mp, s);
    })();
  <% end -%>
</script>

That mpq object looks familiar. We’re testing against it in our integration suite. It’s just a Javascript Array.

We only include the rest of the Mixpanel setup in staging and production. It stuffs mixpanel.js into the DOM asynchronously.

We interpolate our Mixpanel account’s token based on the environment so we can run acceptance on our user story on staging.

config/environments/staging.rb:

MIXPANEL_TOKEN = "our-staging-token".freeze

config/environments/production.rb:

MIXPANEL_TOKEN = "our-production-token".freeze

To get the rest of the integration test passing, we follow the Mixpanel API normally.

views/homes/show.html.erb:

$(function () {
  mpq.push(["track", "visited-home"]);
  $("#tour-cloud").click(function () {
    mpq.push(["track", "clicked-tour"]);
  });
  $("#plans-cloud").click(function () {
    mpq.push(["track", "clicked-plans-and-pricing"]);
  });
  $("#next-to-plans").click(function () {
    mpq.push(["track", "clicked-next-to-plans-and-pricing"]);
  });
});

views/accounts/new.html.erb:

$(function () {
  mpq.push(["track", "viewed-plan", { plan_id: "<%= @plan.id %>" }]);
});

This use case is relatively common. Include some external service’s Javascript and use their Javascript API in order to get good analytics on the app.

To make it happen smoothly, there’s a lot of interpolation and Ruby mixing with HTML and Javascript. Things could easily go wrong and it feels good to have integration coverage for it.

AjaxRecorder: is this insane?

This week, Matt and I were converting pagination in an app to Ajax pagination. In the process, we broke the build.

The failing feature looked something like this:

When I follow "Next page"
And I comment on "Ford" with "Fjord is a better name for a car."
Then I should be on the second page
And I should see the "Fjord is a better name for a car." comment

The implementation of “I should be on the second page” expected that this query string existed: page=2. When we switched to use Ajax, we could no longer check the query string in the same way.

Is this insane?

We came up with the following implementation, and used Akephalos to make sure the Javascript was integrated into the test:

When I follow "Next page"
And I comment on "Ford" with "Fjord is a better name for a car."
Then the comment on "Ford" should have been made via Ajax
And I should see the "Fjord is a better name for a car." comment

It uses this code from features/support/ajax_recorder.rb:

class AjaxRecorder
  def self.clear
    @@records = []
  end

  def self.save(env)
    @@records << env
  end

  def self.has_path?(path)
    @@records.any? { |record| record['REQUEST_URI'] == path }
  end
end

class AjaxRecorderApp
  def initialize(app)
    @app = app
  end

  def call(env)
    if env['HTTP_X_REQUESTED_WITH'] == 'XMLHttpRequest'
      AjaxRecorder.save(env)
    end
    @app.call(env)
  end
end

Capybara.app = AjaxRecorderApp.new(Capybara.app)

Before do
  AjaxRecorder.clear
end

If it’s not clear, we wrote a Rack app that records Ajax requests into the @@records class variable on AjaxRecorder, and inserted it as middleware into our Cucumber/Capybara/Akephalos stack.

We were then able to use AjaxRecorder.has_path? via RSpec in features/step_definitions/ajax_steps.rb:

Then /^the comment on "(.*)" should have been made via Ajax$/ do |value|
  response = fetch_moderated_response(value)
  path     = clients_moderated_response_moderated_comments_path(response)

  AjaxRecorder.should have_path(path)
end

This is testing implementation, not behavior!

One reason this might be insane is that it appears to be testing implementation instead of behavior. Users don’t care about “Ajax”, they care that the site loads the new comment quickly, on the correct page.

How else could we test this?

There are at least two advantages to this test, however:

• We’re confident we’ve tested the Javascript code path, and testing is partly about developer confidence.
• In combination with the “I should see” step checking the state of the page, we’re closer to testing the user experience.

How can we test user experience?

There are certain factors of a web application that are important but a little difficult to capture in Cucumber scenarios. Factors like:

• The app should be fast.
• The content should be organized well.
• I should be able to find and do stuff easily.

Maybe they’re not impossible to capture if we’re willing to use implementation details such as XMLHttpRequest as a proxy for those factors, then tie that implementation to our user experience intent using language in our Cucumber steps:

Then the comment on "Ford" should have been posted quickly, without reloading the page, and with brief visual feedback

“without reloading the page” would be implemented using our AjaxReloader, “posted quickly” could be implemented using a timer/benchmark, and “brief visual feedback” could check that we pulsed/highlighted the new comment.

What do you think? Is this insane?

Use Deadweight and your integration suite to automatically find unused CSS rules

We’re refactoring the CSS in Hoptoad - the app is a few years old, and has been through several rounds of design improvement. We’re looking to combine duplicated rules, make selectors more intention-revealing, and want to reorganize the stylesheets to reflect our current best practices. We’re also switching to SASS, which we’ve found to be a tremendous help. Together, these improvements should cut developer and designer time during maintenance and future design improvements.

Before diving in, we wanted to know what CSS is used or unused.

Deadweight is an excellent tool for identifying unused CSS selectors. It’s flexible, too: you can identify paths in your application for deadweight to scan, or use it as an HTTP proxy and click around in your site, or just pass it static HTML files to analyze. There are a few caveats (for example, selectors inserted by Javascript aren’t tracked) but it’s generally a solid approach for finding unused CSS rules.

I didn’t want to spend a lot of time clicking through the site, and wanted to eliminate the possibility that I would forget a page or two to include. I decided to run the Hoptoad integration tests (Cucumber features, in this case) and capture the response bodies.

Rack middleware to the rescue

I wrote the following piece of Rack middleware:

class ResponseLoggerMiddleware
  RESPONSE_LOG_DIR = Rails.root.join('log', 'responses')
  Dir.mkdir(RESPONSE_LOG_DIR) unless Dir.exist?(RESPONSE_LOG_DIR)

  def initialize(app)
    @app = app
  end

  def call(env)
    response = @app.call(env)
    log(response)
    response
  end

  def log(rack_response)
    code, headers, response = rack_response

    if response.respond_to?(:body)
      html = response.body
      filename = "response_#{Time.now.to_f}.html"
      File.open(RESPONSE_LOG_DIR.join(filename), 'wb') do |file|
        file.puts(html)
      end
    end
  end
end

and included it:

# For Rails 2, include in config/environment.rb
# For Rails 3, include in config/application.rb
# Why Rack::Lock? http://guides.rubyonrails.org/rails_on_rack.html#internal-middleware-stack
config.middleware.insert_after 'Rack::Lock', 'ResponseLoggerMiddleware'

After the tests ran, I processed the output with deadweight:

$ gem install deadweight
$ cat public/stylesheets/*.css | deadweight log/responses/*

Ta-da! If you’re curious what Deadweight output looks like, here’s what I got: https://gist.github.com/cf34709887bf63b1ec15

Further reading

• Ryan Bates’ Railscast on Deadweight will get you familiar with the tool.
• The Helium tool lets you drop a piece of JavaScript on your site and paste in a list of URLs, and will compute unused CSS selectors.
• SitePoint’s Dust-Me Selectors is a Firefox plugin that identify unused CSS selectors. It can spider through your site with a sitemap, too.

Extra credit

You could bundle this all up into a rake task, run it on your CI server, and fail the build whenever there are new unused CSS rules.