AjaxRecorder: is this insane?

This week, Matt and I were converting pagination in an app to Ajax pagination. In the process, we broke the build.

The failing feature looked something like this:

When I follow "Next page"
And I comment on "Ford" with "Fjord is a better name for a car."
Then I should be on the second page
And I should see the "Fjord is a better name for a car." comment

The implementation of “I should be on the second page” expected that this query string existed: page=2. When we switched to use Ajax, we could no longer check the query string in the same way.

Is this insane?

We came up with the following implementation, and used Akephalos to make sure the Javascript was integrated into the test:

When I follow "Next page"
And I comment on "Ford" with "Fjord is a better name for a car."
Then the comment on "Ford" should have been made via Ajax
And I should see the "Fjord is a better name for a car." comment

It uses this code from features/support/ajax_recorder.rb:

class AjaxRecorder
  def self.clear
    @@records = []
  end

  def self.save(env)
    @@records << env
  end

  def self.has_path?(path)
    @@records.any? { |record| record['REQUEST_URI'] == path }
  end
end

class AjaxRecorderApp
  def initialize(app)
    @app = app
  end

  def call(env)
    if env['HTTP_X_REQUESTED_WITH'] == 'XMLHttpRequest'
      AjaxRecorder.save(env)
    end
    @app.call(env)
  end
end

Capybara.app = AjaxRecorderApp.new(Capybara.app)

Before do
  AjaxRecorder.clear
end

If it’s not clear, we wrote a Rack app that records Ajax requests into the @@records class variable on AjaxRecorder, and inserted it as middleware into our Cucumber/Capybara/Akephalos stack.

We were then able to use AjaxRecorder.has_path? via RSpec in features/step_definitions/ajax_steps.rb:

Then /^the comment on "(.*)" should have been made via Ajax$/ do |value|
  response = fetch_moderated_response(value)
  path     = clients_moderated_response_moderated_comments_path(response)

  AjaxRecorder.should have_path(path)
end

This is testing implementation, not behavior!

One reason this might be insane is that it appears to be testing implementation instead of behavior. Users don’t care about “Ajax”, they care that the site loads the new comment quickly, on the correct page.

How else could we test this?

There are at least two advantages to this test, however:

• We’re confident we’ve tested the Javascript code path, and testing is partly about developer confidence.
• In combination with the “I should see” step checking the state of the page, we’re closer to testing the user experience.

How can we test user experience?

There are certain factors of a web application that are important but a little difficult to capture in Cucumber scenarios. Factors like:

• The app should be fast.
• The content should be organized well.
• I should be able to find and do stuff easily.

Maybe they’re not impossible to capture if we’re willing to use implementation details such as XMLHttpRequest as a proxy for those factors, then tie that implementation to our user experience intent using language in our Cucumber steps:

Then the comment on "Ford" should have been posted quickly, without reloading the page, and with brief visual feedback

“without reloading the page” would be implemented using our AjaxReloader, “posted quickly” could be implemented using a timer/benchmark, and “brief visual feedback” could check that we pulsed/highlighted the new comment.

What do you think? Is this insane?