Handle incoming email with Griddler

For all the likes, shares, tweets, pokes, follows, and friends, there’s a fundamental core to the internet that, no matter how hard some might hope, will never go away—email. Rails has built-in support for outgoing mail with ActionMailer, but nothing on the omakase menu handles incoming mail. To help with that, we extracted Griddler from Trajectory and are now happy to release it—hot off the… ahem… presses.

Griddler is a Rails engine that provides an endpoint for the SendGrid Parse API. It hands off a preprocessed email object to a class implemented by you. We’re happy to look at pull requests that interface with other email services.

Setup

To get Griddler integrated with your app, add Griddler to your Gemfile, and bundle away:

gem 'griddler'

Griddler automatically adds an endpoint to your routes table resembling the following:

post '/email_processor' => 'griddler/emails#create'

But you may copy, paste, and modify that anywhere else in your routes for the purposes of your application.

Once Sendgrid posts to your endpoint Griddler will take care of packaging up the important bits of that data and providing a nice Griddler::Email object for you. The contract we expect you to go in on with Griddler at this point is that you will implement a class called EmailProcessor, containing a class method called process, which we will be passing that packaged up instance of Griddler::Email into.

For example, in lib/email_processor.rb:

class EmailProcessor
  def self.process(email)
    # all of your application-specific code here - creating models,
    # processing reports, etc
  end
end

The email object contains the following attributes:

• to
• from
• subject
• body
• raw_body

Of those, to, from, and subject fall on the obvious side as to their purpose.

Let your users interact with your app via email

What isn’t entirely obvious (but very cool) is that Griddler helps you handle the email body by cleaning up replies and providing the important parts of an email before -- Reply ABOVE THIS LINE -- in the body attribute. Note that the reply delimeter is adjustable in the configuration. We keep raw_body around, as contains everything before Griddler scrubs it into body so that you may use the contents for other purposes.

There is much more information in the Griddler README explaining the details, configuration, testing, and other bits.

If you like it, let us know what you think! As always, you can find the code on GitHub. We look forward to hearing all of the ways you use Griddler!

Opinionated settings for app development in Xcode

Setting up a new Xcode project is as simple as ⇧⌘N. Unless you want to do things the right way, at which point there are a number of other configurations you need to worry about: .gitignore, .gitattributes, project level indentation settings, warning levels, etc. After doing the same setup procedure a few times, you can make it a relatively quick process. But it’s 2013, and we’re living in the future now. We have the technology to build tacocopters, but I still have to set my error levels manually? That’s just ridiculous.

Enter: liftoff.

liftoff is a small Ruby gem that we’ve been working on to make new Xcode project setup as fast and painless as possible. With one simple command, you’ll have a slew of defaults set up for your project, along with some things that we think will make your life in Xcode a bit nicer.

Usage

First, install the gem, which is as simple as gem install liftoff. Then, while in your project directory (wherever you are keeping the .xcodeproj file), just run liftoff. This will set up the project defaults we like to use:

• Add the .pbxproj .gitattributes
• Add some sensible patterns to the project .gitignore file
• Set the project indentation settings to 4 spaces
• Set Hosey-level warnings at the project level
• Turn on Static Analysis at the project level
• Treat warnings as errors for release builds
• Add a Run Script build phase to treat TODO and FIXME comments as warnings

These commands can be run individually as well. For example, setting the project’s indentation level to 2 spaces can be done with liftoff indentation 2.

The best part about using liftoff over another solution like a custom project template is that liftoff can be used to quickly add these settings to an existing project without worrying about stomping on your current setup.

liftoff is currently at version 0.6 and, as usual, the code is open source on GitHub. We hope you enjoy it as much as we do.

Hidden gems: suspenders

Have you seen our suspenders? It is a Rails app generator that includes a set of Ruby gems and common code that we use on every project.

Install it

gem install suspenders

This will install gems with C code, which means you need your OS set up for that. On Debian you need rubygems and build-essential; on OS X you can use the laptop script.

Suspenders will also install capybara-webkit by default. Read the instructions for setting up your OS for it, or use the --webkit false option with the following command.

Create a Rails app

suspenders demo

This makes a demo app. See the difference between it and a bare Rails app.

Gems

The gems are a mix of production libraries like newrelic, development libraries like foreman, and testing libraries like timecop. The full list often changes, so see templates/Gemfile_additions for the exact list.

We selected these gems because we use them on most apps, they have tests, and they are maintained. Some of them are ours but most are from others in the community.

Common code

Suspenders includes a Guardfile, flash and Javascript partials, configuration for Heroku, Factory Girl, Bourbon, Postgres, and more:

• An email address validator so you can do validates :login, email: true
• A list of HTTP and SMTP exceptions that we have gathered through trial and error.

• Prefilled form inputs using JavaScript for a nice user experience:

  <input type="text" class="prefilled" title="Email" name="user[email]" />

Even better Suspenders

Create Heroku staging and production apps with the --heroku option:

suspenders demo --heroku true

Create a Github project with the --github option:

suspenders demo --github organization/project

Use a different authentication library or no auth at all with the --clearance option:

suspenders demo --clearance false

Opt out of the capybara-webkit dependencies with the --webkit option:

suspenders demo --webkit false

Or, combine options:

 suspenders demo --heroku true --github organization/project --webkit false

That was easy

Suspenders should make your life easier. Reduce the tedious decisions, add the one-liners you wish you had, and set up the infrastructure common to every codebase.

Use suspenders for your next Rails app!

Hidden gems: paul_revere

There are times when you must proclaim from the rooftops, for all to hear, that your Web site has something to say! “Hear ye! Hear ye! The Swedish are coming!”, just like Paul Reveresson said.

Thus we have the paul_revere gem.

Install it

To start, toss the gem in your Gemfile:

gem 'paul_revere'

There is an included generator, so be sure to run it:

rails g paul_revere

This will generate a migration to add an annoucements table, so run that:

rake db:migrate

Users of older versions of Rails will see a generated JavaScript file too.

We’re almost there. Next up modify your layout to have a place for announcements. In app/views/layouts/application.html.erb:

<%= render 'announcements/announcement_for_all' %>

And finally add the JavaScript to the asset pipeline. In app/assets/javascripts/application.js:

//= require announcements

Whew! A simple process that will only become easier with improvements to Rails.

Use it

% rails c
> Announcement.create!(body: "I'm as mad as hell and that's OK!")

That was easy

As you just saw, using it is straightforward. This builds on an old RailsCast dealing with this topic. With these pieces you can build more complex tools into your app: an admin section that can make announcements, a dashboard where users can see prior announcements, a record of how many announcements you make per month, and so on.

Don’t re-invent the wheel; use the paul_revere gem today!

Get Your C On

This year, whyday happened to fall on the first day of Capeco.de. I’d been interested in playing with C for a while, so I decided to sit down and start learning. The K&R helped a lot, but digging through both Redis and Potion were a lot more insightful of how C could actually be used.

Since then, I’ve been trying to get some C in my Ruby by writing gems with C extensions. This isn’t hard to do, per se, but it was difficult to find a comprehensive list of requirements (as well as guidelines for organizing the code). I ended up looking at one of the most-used Ruby gems with C - Nokogiri.

Where to Start

You’ll want a way to test the code, as well as the rake-compiler gem. I don’t test my C explicitly - think of it as a handful of private methods. You’ll want to test the public methods on whatever classes and modules you write, so a C testing framework is overkill. Finally, you’ll want to become acquainted with ruby.h.

I decided to start simple and write a Sieve of Eratosthenes. I’d written one in pure Ruby and wanted a basic translation to C. I was also interested in benchmarking the code since I knew C would be a lot faster in this instance.

The sieve gem can be found here and its source here.

Directory Structure

A C extension’s directory structure is very similar to other Ruby gems; the only addition is an ext directory that will store files necessary for generating a Makefile and compiling the code. This is where your C files and their headers will go. In my case, these files are located in ext/sieve.

extconf.rb

extconf.rb is what will generate your Makefile. You’ll need to require "mkmf" and then call create_makefile("your_gem/your_gem"). The mkmf documentation is an excellent resource if you want to include other libraries or customize anything. My gem is straightforward so all I did was ensure the Makefile was created.

sieve.h

My sieve.h is very straightforward and doesn’t really need any explanation.

sieve.c

This is the meat and potatoes of the gem.

At the top of the file, you’ll need to #include <ruby.h> as well as any other headers you need.

You’ll also need an Init_your_gem() function that will be called similarly to main(). This is where I create the structure of my classes and modules for my sieve. I create a Sieve module, add a sieve instance method, and then have the Numeric class include Sieve.

Finally, there’s the sieve function itself. It returns a Ruby object which is of type VALUE. It also accepts a Ruby object (self), which is also of type VALUE. An important reminder is to make sure you free any memory you allocate or your gem will leak memory, just as you would in C.

The /lib directory

Although most of the actual work is done in C, we’ll want to have a bit of Ruby in the /lib directory. I’ve written a scaffold of the module at lib/sieve.rb, which has a require "sieve/sieve" at the top of the file (remember in extconf.rb when we passed a string to create_makefile? That’s it.).

The Rakefile

Being able to compile the gem and run the tests is important, which is why I mentioned the rake-compiler gem earlier. After requiring rubygems, rake, and your library, you’ll want to require "rake/extensiontask". That’ll give you a couple of handy rake tasks, namely clean and compile.

I like to set my default task to run tests, but you’ll want a couple prerequisites to that task: clean and compile. This will ensure that you’re rebuilding your gem and running with the latest compiled version.

Since I’m using Cucumber, it looks like this:

require "cucumber/rake/task"
Cucumber::Rake::Task.new(:cucumber => [:clean, :compile]) do |t|
  t.rcov = true
end

task :default => :cucumber

I also have my benchmark task here so I can find out how much more performant this library is compared to a pure Ruby implementation.

Testing

You’ll want to test your C extension just like any other Ruby gem. I prefer Cucumber but anything will do. I used a scenario outline for some of the basic primes and then found a file of the first one million primes for some heavy-duty lifting. I also tested that if enough memory couldn’t be allocated, it would raise a Ruby NoMemoryError exception.

Since you’re going to want to run the features against the latest changes of your gem (and not a version of the gem that’s installed), you’ll want to modify the load path within your test helper (test/test_helper.rb, spec/spec_helper.rb, or features/support/env.rb). My env.rb looks like this:

$LOAD_PATH.unshift(File.dirname(__FILE__) + '/../../lib')
require "sieve"
require "spec/expectations"

Building the gem

The gemspec for a Ruby C extension is fairly straightforward. The only thing you’ll need to add is to set the spec’s extensions attribute to the path to the extconf.rb file.

Gem::Specification.new do |s|
  s.require_paths = ["lib"]
  s.extensions = ["ext/sieve/extconf.rb"]
  # ... rest of the gemspec
end

As with any gemspec, you’ll want to make sure that you list the .c and .h files within files.

Results

Armed with this, you should be able to go and write Ruby C extensions to your hearts content. As for my Sieve experiment, here’s the pure-Ruby implementation of the sieve:

# usage:
#   >> sieve 100
#   => [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]
def sieve(n)
  numbers = (0..n).map {|i| i }
  numbers[0] = numbers[1] = nil
  numbers.each do |num|
    next unless num
    break if num**2 > n
    (num**2).step(n, num) {|idx| numbers[idx] = nil }
  end
  numbers.compact
end

My benchmarks were running the sieve on numbers from zero to one million in steps of 100,000. No memoization is used for either form.

On Ruby 1.8.7, here are the results from my benchmark:

                   user     system      total        real
sieve method   4.460000   0.060000   4.520000 (  4.522069)
Numeric#sieve  0.040000   0.000000   0.040000 (  0.046349)

Ruby 1.9.2 is significantly faster, but still doesn’t hold a candle to the C extension:

                   user     system      total        real
sieve method   2.410000   0.060000   2.470000 (  2.468430)
Numeric#sieve  0.050000   0.000000   0.050000 (  0.049053)

What I Learned

Writing C is both fun and can enhance performance of number-crunching and other fun things. It has it’s place and is a great addition to any Rubyist’s toolbox. Have you written any C extensions purely for performance gains? If you’re open to sharing the context, I’d love to hear about it!