Recipe: Ajax Searching And Filtering

A recipe for adding searching and filtering to your Rails app.

Why

Provide faster feedback for users. Increase the chance that they’ll find what they want.

Feature

This feature is testing the non-JavaScript path through the app. I’m not going to show any JavaScript testing in this recipe.

Scenario: Search by fieldnotes
  Given a project exists with a name of "Big Dig"
  And the following reports exist:
    | name            | fieldnotes | project       |
    | Traffic Control | Barricade  | name: Big Dig |
    | Gases, Vapors   | Dust       | name: Big Dig |
  When I sign in as a safety manager of "Big Dig"
  And I go to the "Traffic Control" report page
  And I search for "Dust"
  Then I should be on the reports page
  And I should see "Gases, Vapors"
  And I should not see "Traffic Control"
  And the search field should contain "Dust"

I know I’m going to be writing multiple search scenarios so I created some step definitions for search:

When /^I search for "([^\"]*)"$/ do |query|
  When %{I fill in "search-field" with "#{query}"}
  And %{I press "Search"}
end

Then /^the search field should contain "([^\"]*)"$/ do |query|
  field_with_id("search-field").value.should == query
end

I know I want search-field because the designer has already sliced the HTML and CSS.

HTML

The search box:

- form_tag reports_path, method: :get do
  = text_field_tag 'query', params[:query], id: 'search-field'
  = submit_tag 'Search', id: 'search-reports-button'

Only thing that might be of note is using params[:query] to get the ‘And the search field should contain “Dust”‘ test to pass.

The filters:

#filter-list{ style: 'display: none;' }
  %form#filters
    / a bunch of checkboxes

The results:

%tbody.striped#reports-tbody
  = render partial: 'report', collection: @reports

Controller

The index action:

def index
  @reports = Report.search(params)

  if request.xhr?
    render partial: '/reports/report', collection: @reports
  end
end

There’s some authorization around scoping results to the current user’s project that I’ve removed for brevity.

Model specs

There’s about a dozen different ways this data can be filtered. There’s specs for each kind that look something like this:

it 'finds reports assigned to any of a set of safety inspectors' do
  first = create(:safety_inspector)
  second = create(:safety_inspector)
  third = create(:safety_inspector)
  find_me = create(:report, safety_inspector: first)
  me_too = create(:report, safety_inspector: second)
  not_me = create(:report, safety_inspector: third)

  reports = Report.search(safety_inspectors: "#{first.id},#{second.id}")

  expect(reports).to include(find_me)
  expect(reports).to include(me_too)
  expect(reports).to_not include(not_me)
end

There’s one for sending a text query, and a giant one that tests combinations.

Searchlogic

The method through which all searching and filtering passes:

def self.search(params)
  full_text_search(params[:query].to_s).
  location_in(params[:locations].to_s).
  safety_inspector_in(params[:safety_inspectors].to_s).
  safety_category_in(params[:safety_categories].to_s).
  created_after(params[:from].to_s).
  created_before(params[:to].to_s).
  severity_in(params[:severities].to_s).
  state_in(params[:states].to_s).
  descend_by_severity_and_created_at.
  distinct
end

Most items in this chain are custom class methods that wrap Searchlogic:

def self.location_in(locations)
  if locations.blank?
    scoped
  else
    location_id_equals_any(locations.to_array_of_ints)
  end
end

def self.distinct
  select("distinct reports.*")
end

The scoped is a way of maintaining chainability in the common cases where most searching and filtering criteria is blank.

The array of integers is a custom String extension:

class String
  def to_array_of_ints
    split(',').map { |integer| integer.to_i }
  end
end

The idea here is to do all filtering on highly indexed integers, which Postgres handles quickly. It’s also easy to pass in comma-separated ids from jQuery as we’ll see later.

“Full text search” is just SQL ILIKEing while avoiding SQL injection:

def self.full_text_search(query)
  if query.blank?
    scoped
  else
    text_search(query)
  end
end

def self.text_search(query)
  join_sql = <<-SQL
    INNER JOIN locations ON locations.id = reports.location_id
    INNER JOIN users ON users.id IN (
      reports.safety_inspector_id,
      reports.supervisor_id,
      reports.subcontractor_id
    )
  SQL

  condition_sql = <<-SQL
    (reports.fieldnotes ILIKE :query) OR
    (reports.name ILIKE :query) OR
    (locations.name ILIKE :query) OR
    (users.name ILIKE :query)
  SQL

  joins(join_sql).where(condition_sql, { query: "#{query}%" })
end

jQuery

Bind all the necessary events:

$(document).ready(function() {
  $('#clear-filters-btn').click(function() {
    $('#filters :checked').attr('checked', false);
    $('#slider').slider('values', 0, $('#slider').slider('option', 'min'));
    $('#slider').slider('values', 1, $('#slider').slider('option', 'max'));
    searchReports();
  });

  $("#filter-list-btn").click(function(){
    $(this).toggleClass("active");
    $("#filter-list").slideToggle("500");
    return false;
  });

  $('#search-field').keyup(searchReports);

  $('#filters :checkbox').click(searchReports);
  $('#filters :text').focus(searchReports);
});

Build the Ajax call with jQuery.param and call it:

function searchReportsURL(){
  var params = {
    'query' : escape($('#search-field').val()),
    'locations' : checkedIdsForFilter('location'),
    'supervisors' : checkedIdsForFilter('supervisor'),
    'safety_categories' : checkedIdsForFilter('category'),
    'severities' : checkedIdsForFilter('severity'),
    'states' : checkedIdsForFilter('state'),
    'risk_profiles' : checkedIdsForFilter('risk-profile'),
    'from' : $('.left-handle').text(),
    'to' : $('.right-handle').text()
  }

  return '/reports?' + $.param(params) + '&' + (new Date()).getTime();
}

var searchReportsTimeout = null;

function searchReports() {
  if (searchReportsTimeout) {
    clearTimeout(searchReportsTimeout);
  }

  searchReportsTimeout = setTimeout(function() {
    $.get(searchReportsURL(),
      function(data) {
        $('#reports-tbody').html(data);
        $(document).trigger('stripeRows');
      }
    );
  }, 500);
}

For brevity, I’ve omitted some helpers like checkedIdsForFilter that build a comma-separated list of ids for each criteria. You can figure that out based on your own markup and JavaScript’s replace() method.

I kept the timeout, however, because I think it’s important for the user experience. Without it, the search will happen too fast on every keyup(), resulting in a herky-jerky experience. There’s a half-second pause now, but that’s preferred over “strobe-lighting” the user.

Appending the date to the end of the URL is a cache-buster for Internet Explorer.

Bon appétit!