Testing Cron on Heroku

Deploying cron to Heroku is really…pleasant. Click “Daily” or “Hourly” cron and without any tedious setup of scripts, ensuring output is logged, or referring to CronWTF. Testing it though, is a pain! No longer should this be the case.

Cron is serious business.

Outside of Heroku, splitting up daily and hourly cron tasks is easy: just have a script/cron_hourly and script/cron_daily in your Rails app, and have fun configuring that on your server. On Heroku, it’s handled in one Rake task. Here’s an example from the Dev Center:

desc "This task is called by the Heroku cron add-on"
task :cron => :environment do
  if Time.now.hour % 4 == 0 # run every four hours
    puts "Updating feed..."
    NewsFeed.update
    puts "done."
  end

if Time.now.hour == 0 # run at midnight
    User.send_reminders
  end
end

Two things stand out here. First, checking for hourly/daily tasks is done by looking at Time.now. Second, that’s a lot of logic to put in a Rake task, and not write a test!

We’ve talked about testing Rake integration before, and we’re going to use a similar pattern here: extract the Rake task into a model. For Radish, our Rakefile now has:

desc "Run cron job"
task :cron => :environment do
  Cron.run
end

Our Cron class now has to handle the daily and hourly tasks. For now, this is all done in the run class level method. For Radish, the method has to:

• Archive data hourly for our new historical graphs
• Activate accounts daily, which will prevent a reminder email from being sent out.

The test

I ended up just using RSpec to test this out. Timecop helps with freezing time in the right place, and Bourne gives us test spies to make sure the right methods get called. Here’s the test I ended up with:

require 'spec_helper'

describe Cron do
  before do
    Account.stubs(:activate)
    Archive.stubs(:store)
  end

  let!(:project1) { Factory(:project) }
  let!(:project2) { Factory(:project) }

  after do
    Timecop.return
  end

  it "runs nightly" do
    Timecop.freeze(Time.now.midnight)
    Cron.run

    Account.should have_received(:activate)
    Archive.should_not have_received(:store)
  end

  it "runs hourly" do
    now = Time.now.midnight + 1.hour
    Timecop.freeze(now)
    Cron.run

    Account.should_not have_received(:activate)
    Archive.should have_received(:store).with(project1, now, now - 1.hour)
    Archive.should have_received(:store).with(project2, now, now - 1.hour)
  end
end

Let’s start from the top of the test here. The before block uses Bourne to stub out the two class level methods on other models in the application, so we can assert they were called later. We then hook up two let! blocks for two projects, which we will use later. let! as opposed to let in RSpec will force those blocks to be evaluated for each test run instead of being lazy evaluated when they are referenced. Finally, since we’re going to freeze time for each test, we have to return to the system time in the after block.

Our two tests verify our daily and hourly scenarios. The first freezes time at midnight, which may not be exactly when Heroku runs our cron job, but all we care about is that the daily task runs only once. The hourly test freezes time at 1:00AM and checks that only the hourly task gets run, and not the daily.

I could have gone a little more gung-ho on this test, perhaps running through an entire 24 hours and making sure the daily task was only called once, but this was good enough.

The implementation

Here’s what I ended up with in my Cron model:

class Cron
  def self.run
    now = Time.now

    Project.find_each do |project|
      Archive.store(project, now, now - 1.hour)
    end

    if now.hour == 0
      Account.activate
    end
  end
end

The implementation ended up to be pretty simple: grab the time, archive always (since the task is run hourly), and if it’s run in the 12:00AM hour, activate accounts.

Pushing this code down into a model makes more sense now…too much code that deals with models and not test data or factories in a Rake task always smells a bit funky to me. It’s also much easier to refactor the code now that we’re in a real model and we have a testing feedback loop in place. For instance, the Project.find_each loop could easily be extracted into the Project class.

Hacking your time zone

During this process I learned of a UNIX trick that can help with testing this locally: the TZ flag. The appropriately named UNIX Power Tools puts it best:

The TZ environment variable is a little obscure, but it can be very useful. It tells UNIX what time zone you’re in.

Most of the time scripts will get this from your environment, but you can override it. Here’s a simple way to test this:

% TZ=UTC+5 ruby -e "puts Time.now"
2011-07-05 11:30:48 -0500

% TZ=UTC-8 ruby -e "puts Time.now"
2011-07-06 00:30:42 +0800

% TZ=UTC ruby -e "puts Time.now" 
2011-07-05 16:30:53 +0000

So basically, if you want to force it to be midnight when running a test or from a small script, you can use this environment variable to add/subtract time from your current time zone.

Less of a pain

Testing cron is now actually feasible, and now you can be assured your task will work without waiting an entire hour or day to find out. Which of course, means you can ship it faster!

Do-it-yourself @font-face web service

Phil recently designed Developers Developers Developers Developers, an event we’re running for Boston-area high school and college students:

It looks awesome in part because of liberal use of @font-face.

These days, it’s simple to use @font-face. Just use a stylesheet referencing a web service like Google Font Directory:

 <link href='http://fonts.googleapis.com/css?family=Lobster&subset=latin' rel='stylesheet' type='text/css'>

… then use the font as part of a normal CSS font stack:

#schedule .event-time {
  font-family: 'Lobster', arial, sans-serif;
}

What if the font isn’t on an @font-face web service?

There are plenty of commercial @font-face web services, which Kyle has covered in detail.

However, there are cases where the font you want to use isn’t on an existing service. Or, the font you want to use requires explicit permission from the font’s creator before you can use it.

We ran into this problem on Developers Developers Developers Developers.

First attempt: serve fonts from the app

To solve it, we included the font files in our git repository. This worked, but wasn’t ideal:

• Unnecessarily adds large files to the repo.
• No HTTP caching by default, downloading font files on subsequent requests.
• Repetition. Soon after, I wanted to use League Gothic again on another app.

Second attempt: serve fonts directly from Amazon S3

So our next attempt was to upload the @font-face stylesheet and font files to S3 and serve them directly from there like:

<link href='http://our-bucket.s3.amazonaws.com' type='text/css'>

At first, this seemed to work well. We could set some HTTP headers to handle caching and everything seemed like it was in the right place:

• application code in version control
• assets in an asset host (S3)

However, Firefox (and probably some versions of IE), balks at this technique because of the same origin policy.

On S3, you’re not allowed to tweak the Access-Control-Allow-Origin HTTP header to allow Firefox to serve your font from S3. There’s a long Amazon thread where S3 customers are asking for this ability.

However, even if you could set the Access-Control-Allow-Origin header on S3, you also want the Content-Type, Cache-Control, and Expires to all also be set in a standard way. It’s a pain to have to do that manually.

Solution: serve fonts from a DIY web service

Our final solution was a small Sinatra app called Fist Face. It is now open source under the MIT License and it solves all the problems we experienced.

Fist Face

It works exactly the same way as Google Font Directory, Typekit, or any other @font-face web service, except that you have full control over it:

<link href='http://your-font-face-service.com/league-gothic.css' rel='stylesheet' type='text/css'>

To use Fist Face, this is all you have to write:

# Gemfile
source 'http://rubygems.org'
gem 'sinatra',  '~> 1.1'
gem 'fistface', '~> 1.0'

# config.ru
require 'rubygems'
require 'bundler'
Bundler.require
run FistFace

Then deploy it. Follow a few conventions in the README regarding your asset host (ex: S3) and a few minutes later, you’ll be serving fonts via your own @font-face web service.

Why would I want to do it myself?

If you’ve run into the same issues that we did, have you solved this problem differently?

Among the weaknesses of this approach are:

• No CDN. (patches welcome!)
• Up to you to write a decent cross-browser CSS file.
• The font files you use might be old-school (not “hinted” for computer screens).
• Requires deeper knowledge of typefaces.

With an open mind, the last weakness is actually a strength. Typography is pretty interesting. You’ve got all these independent type foundries doing beautiful work and some release their fonts for free under permissive licensing.

In that way, hunting great typefaces for your @font-face web service is like building your own art collection. It can differentiate your work.

So, when your friends ask you, “whoa, what font is that?”, you can tell them, “you’ve probably never heard of it” … which I’ve heard is hip.

Written by Dan Croak.

Hoptoad is now available as a Heroku Addon

We’ve been working with Heroku to bring you an even easier and more convenient way to add Hoptoad to your Heroku apps:

$ heroku addons:add hoptoad:plus

• $7/month, SSL, ratelimit of 30 errors/minute, plus Github, Lighthouse, and deploy integration

$ heroku addons:add hoptoad:basic

• $2/month, SSL, ratelimit of 5 errors/minute

Or, if you prefer, you can add Hoptoad to your app with a single click over on the Hoptoad Heroku Addon page.

Once you’ve configured the addon, you’ll want to install the Hoptoad Notifier in your project, and you can use a shiny, new, Heroku-specific configuration flag in the generator:

$ rails generate hoptoad --heroku

Then, you can click through from your Heroku dashboard to the Hoptoad addon, and get the same Hoptoad interface you’re come to know and love:

The Heroku Addon Experience

I would highly recommend building your own Heroku addon. The experience is getting very polished, and there is a large amount of documentation at the Heroku Addon Provider site. They take care of billing and provide a single-sign on API for addon providers, and generally make the process seamless and pain-free. We’ve been working with them for a while, through the development of their addon platform, and they have a very solid offering now. Go forth and develop!

Video: Sinatra at Boston.rb, part 1

This the first in a series of short videos. They feature Blake Mizerany discussing Sinatra and Heroku in great technical detail at September’s Boston.rb.

Blake Mizerany wrote Sinatra in 2006 because he was working on a high traffic site with a lot of POSTs, PUTs, and DELETEs. GETs can be cached but the others cannot.

In this video, Blake discusses the following concepts as they apply to Sinatra:

backing into patterns

Start with a string, start with a text file, use main on Ruby. Move up to Factories and Model-View-Controller and ActiveRecord when you need it.

Rails creates 72 files when you run the rails command. Blake decided he didn’t need to start with MVC on each new project.

clean routing system

Blake uses Sinatra often for web services. He compares Rails’ respond_to with Sinatra’s content_type declaration. He discusses his feeling that the routing systems in other frameworks are overly complex undesirable.

get '/users.json' do
  content_type :json
  @user = User.find(params[:id])
  @user.to_json
end

His first example, shown above, highlights Sinatra’s clean content type approach.

next

In the next video, Blake discusses:

• handling the Accepts header using Rack middleware
• deciding between Rack and Sinatra
• rack-hoptoad
• routing tricks such as passing wildcard params into block variables and non-greedy matching