Handle incoming email with Griddler

For all the likes, shares, tweets, pokes, follows, and friends, there’s a fundamental core to the internet that, no matter how hard some might hope, will never go away—email. Rails has built-in support for outgoing mail with ActionMailer, but nothing on the omakase menu handles incoming mail. To help with that, we extracted Griddler from Trajectory and are now happy to release it—hot off the… ahem… presses.

Griddler is a Rails engine that provides an endpoint for the SendGrid Parse API. It hands off a preprocessed email object to a class implemented by you. We’re happy to look at pull requests that interface with other email services.

Setup

To get Griddler integrated with your app, add Griddler to your Gemfile, and bundle away:

gem 'griddler'

Griddler automatically adds an endpoint to your routes table resembling the following:

post '/email_processor' => 'griddler/emails#create'

But you may copy, paste, and modify that anywhere else in your routes for the purposes of your application.

Once Sendgrid posts to your endpoint Griddler will take care of packaging up the important bits of that data and providing a nice Griddler::Email object for you. The contract we expect you to go in on with Griddler at this point is that you will implement a class called EmailProcessor, containing a class method called process, which we will be passing that packaged up instance of Griddler::Email into.

For example, in lib/email_processor.rb:

class EmailProcessor
  def self.process(email)
    # all of your application-specific code here - creating models,
    # processing reports, etc
  end
end

The email object contains the following attributes:

• to
• from
• subject
• body
• raw_body

Of those, to, from, and subject fall on the obvious side as to their purpose.

Let your users interact with your app via email

What isn’t entirely obvious (but very cool) is that Griddler helps you handle the email body by cleaning up replies and providing the important parts of an email before -- Reply ABOVE THIS LINE -- in the body attribute. Note that the reply delimeter is adjustable in the configuration. We keep raw_body around, as contains everything before Griddler scrubs it into body so that you may use the contents for other purposes.

There is much more information in the Griddler README explaining the details, configuration, testing, and other bits.

If you like it, let us know what you think! As always, you can find the code on GitHub. We look forward to hearing all of the ways you use Griddler!

Delivering all email from staging to a group email address

All email from the staging environment of a Rails app can be intercepted and delivered to a group email address. This avoids accidentally delivering staging email to production customers and lets the product team see all the emails that are being sent to customers.

Configure

Gemfile:

gem 'recipient_interceptor'

config/initializers/mail.rb:

if Rails.env.staging? || Rails.env.production?
  MAIL_SETTINGS = {
    address: 'smtp.sendgrid.net',
    authentication: :plain,
    domain: 'heroku.com',
    password: ENV['SENDGRID_PASSWORD'],
    port: '587',
    user_name: ENV['SENDGRID_USERNAME']
  }
end

config/environments/{staging,production}.rb:

require Rails.root.join('config/initializers/mail')

My::Application.configure do
  config.action_mailer.delivery_method = :smtp
  config.action_mailer.smtp_settings = MAIL_SETTINGS
end

Also in config/environments/staging.rb:

Mail.register_interceptor RecipientInterceptor.new(ENV['EMAIL_RECIPIENTS'])

Use the ENV['EMAIL_RECIPIENTS'] environment variable to update the list of email addresses that should receive staging emails. For example:

heroku config:add EMAIL_RECIPIENTS="staging@example.com" --remote staging

Gmail filter

Depending on the app, this may generate thousands of emails a day. Avoid spamming yourself by setting up a Gmail filter for emails sent to staging@example.com.

Written by Dan Croak.

Using Writing Smells to Refactor Your Email

I’ve recently started applying some code refactoring techniques to writing emails. Yes, emails.

Your goal as the writer

1) Get the reader to read the most important thing
2) Get them to respond quickly or do something quickly

We all get too much email and the following is my approach to try and make emails simpler, easier to respond to, and more readable.

I adapt the concept of code smells for use in email as “writing smells”. Here’s my process and an example. It’s not perfect but it’s useful for me.

Writing Smells

Forest for the Trees

Break every sentence into a separate paragraph (each on a new line with a blank between). This helps to highlight long sentences. Also, it narrows your refactoring focus to one sentence at a time and improves readability.

More Than One “and”

Break long sentences down into shorter ones, especially if they have two ideas/goals. Example offender:

“If you haven’t already, take a look at our site http://thoughtbot.com for some examples of projects we’ve done and our playbook (http://playbook.thoughtbot.com) for more detail on our process and approach to building products”

Sentences > 20 Words

Start at the top and focus on one sentence at a time. Play Sentence Golf - use as few words as possible and still be clear.

Now What?

Your goal as an email sender is to convey information and get some kind of action to take place. If someone reads your message and it doesn’t address what they should do, it’s going to get ignored.

First Sentence Doesn’t Convey Everything

The most important thing for the reader to pay attention to should be at the top. Most email readers only get to the first sentence - make it count.

Open-ended Choices

You should offer the recipient only one decision and it should be a “yes/no” choice. If your question requires work, you have a lower chance of getting a response. For example, when proposing to meet up, say

“I like Cafe Madeline on New Montgomery. Let’s meet there Tuesday at 4pm.”

instead of

“Where should we meet?”

If they don’t like the date/time, they will propose something else.

More than 5 Sentences

Keep it to 5 sentences or fewer if you can.

Meh

Throw a joke in there. Email sucks and we have tons of it. Amuse your reader somehow.

And I Know You From…?

If it’s your first message to someone, demonstrate how you know each other, “We met at last night’s meetup”. Throw in some connection like “We are both from Quebec!”

Kinda Sorta

Avoid being wishy washy. Be direct. For example,

“If you’re still going to be in town next week I’d be glad to meet with you and talk about what you guys are doing at example.com, what we do here at thoughtbot and how we could help you out.”

becomes

“If you’re in town next week, let’s meet up. I suggest Cafe Madeline…”

Overuse

I overuse the word ‘that’. When finishing an email, I do a Ctrl-F on ‘that’ and either delete it or replace it with ‘which’ when appropriate. Or use a tool to find words you tend to overuse.

Like the Plague

Avoid cliches. Like the plague.

Here’s an example

Before:

“Hey Ralph,

I’m glad I got to meet you last night. If you’re still going to be in town next week I’d be glad to meet with you and talk about what you guys are doing at example.com, what we do here at thoughtbot and how we could help you out. If you haven’t already, take a look at our site http://thoughtbot.com/ for some examples of projects we’ve done and our playbook (http://playbook.thoughtbot.com/) for more detail on our process and approach to building products. If you are interested I can also send you a document which provides an overview of our process. Looking forward to hearing from you. -Rolf”

After:

“Hey Ralph,

I’m glad to meet another fellow Quebecois and lover of poutine last night at the meetup.

If you’re in town next week, let’s meet up. I suggest Cafe Madeline for coffee on Tuesday at 3pm.

I’m interested to learn more about example.com and tell you more about thoughtbot.

If you haven’t already done so, take a look at thoughtbot.com to see some of our recent projects.

Also, take a look at our playbook (playbook.thoughtbot.com) for more detail on our approach to building startups.

Looking forward to hearing from you.

-Rolf”

Further reading:

• Guy Kawasaki’s “The Effective Emailer”
• Shell Scripts to Improve Writing

Photo: Skunk Encounter by vladeb, on Flickr

Delivering email on behalf of users

A recipe for a better user experience in emails sent between users via my Rails app.

Why?

When I receive an email from an automated system like a Rails app, it is disorienting if the sender shows up in my email program as “admin” or “donotreply”.

What I want is something like this:

Ingredients

• ActionMailer
• The email_spec Ruby gem

Install email-spec

I’m a fan of Ben Mabey’s email-spec gem, so I’ll install that:

group :test do
  gem 'email_spec'
end

I create a features/support/email.rb file:

require 'email_spec' # add this line if you use spork
require 'email_spec/cucumber'

Then generate some step definitions into features/step_definitions/email_steps.rb:

rails generate email_spec:steps

Feature

Now I’ll write my user story:

Scenario: Guitarist shares song with guitarist
  Given the following user exists:
    | name         | email            |
    | Eric Clapton | eric@example.com |
  And I sign in as "eric@example.com/password"
  And I am on the share page for "Layla"
  When I fill in "Share with" with "jimi@example.com"
  And I press "Share Song"
  And "jimi@example.com" opens the email
  Then he should see "Eric Clapton " in the email "From" header
  And he should see "eric@example.com" in the email "Reply-To" header

The “From” and “Reply-To” headers

I think the “From” and “Reply-To” headers can provide a better user experience.

I don’t set the author’s email as the “From” header because I hear it’s bad spam practice to send email on behalf of users in that way. ISPs use the From header (among other things) to determine if the originator is sending spam.

Making the feature pass

Ease my worried mind:

class Mailer < ActionMailer::Base
  def share_song(song, friend)
    mail(
      to: friend.email,
      from: %{"#{song.artist.name}" },
      reply_to: song.artist.email,
      subject: 'Good song'
    )
  end
end

I’ve used this format so the sender’s name shows up in the receiver’s email program:

"Name" 

In this case, I want Jimi to be able to reply directly to Eric, so I’ve set the “Reply-To” header to be the sender’s address. I’ve explicitly not put the sender’s name in the “Reply-To” header because that doesn’t work.

In other cases, I want the receiver to reply to the email and have that sent through the Rails app, but that’s a story for another day.

Written by Dan Croak.

I accidentally the whole SMTP exception

You have a slick, exclusive, invite-only Web app for sharing Tor URLs, with an Android client and specialty hardware. You use validates_email_format_of in the Invitation model, but still something slips through and your Hoptoad errors pile up, showing your user the beautifully-designed 500 page instead of an error explanation.

There are two types of exceptions that ActionMailer will raise when you attempt to deliver an email: user input problems and server problems.

User input problems are those such as incorrect or invalid email addresses; the exceptions raised are Net::SMTPFatalError and Net::SMTPSyntaxError. These are issues that the user can fix and as such the error message should indicate that everything’s fine, nothing is ruined.

Server problems could be anything from a non-existent server to an authentication issue; the exceptions raised are: TimeoutError, IOError, Net::SMTPUnknownError, Net::SMTPServerBusy, and Net::SMTPAuthenticationError. These issues are outside the power of the user and should indicate that we screwed up.

So in config/initializers/errors.rb:

SMTP_SERVER_ERRORS = [TimeoutError,
                          IOError,
                          Net::SMTPUnknownError,
                          Net::SMTPServerBusy,
                          Net::SMTPAuthenticationError]

SMTP_CLIENT_ERRORS = [Net::SMTPFatalError, Net::SMTPSyntaxError]

SMTP_ERRORS = SMTP_SERVER_ERRORS.concat(SMTP_CLIENT_ERRORS)

SMTP_CLIENT_ERROR_FLASH = 'The email address supplied is invalid.  Please check for spelling mistakes.'
SMTP_SERVER_ERROR_FLASH = 'We encountered an internal issue while attempting to deliver this email.  Please try again in a few minutes.'

We can test it with invitations_controller_test.rb:

class InvitationsController; def rescue_action(e) raise e end; end
class InvitationsControllerTest < Test::Unit::TestCase
  SMTP_CLIENT_ERRORS.each do |exn|
    should "handle #{exn}" do
      InvitationsMailer.expects(:deliver_invitation).raises(exn)
      post :create, :invitation => {:email => 'invalid email'}
      assert_match /#{SMTP_CLIENT_ERROR_FLASH}/i, @response.flash[:warning]
      assert_template 'new'
    end
  end

  SMTP_SERVER_ERRORS.each do |exn|
    should "handle #{exn}" do
      InvitationsMailer.expects(:deliver_invitation).raises(exn)
      post :create, :invitation => {:email => 'invalid email'}
      assert_match /#{SMTP_SERVER_ERROR_FLASH}/i, @response.flash[:warning]
      assert_template 'new'
    end
  end
end

And in invitations_controller.rb:

class InvitationsController < ApplicationController
  def create
    @invitation.new(params[:invitation])
    if @invitation.save
      redirect_to root_url
    else
      render :action => 'new'
    end
  rescue *SMTP_CLIENT_ERRORS
    flash[:warning] = SMTP_CLIENT_ERROR_FLASH
    render :action => 'new'
  rescue *SMTP_SERVER_ERRORS => error
    notify_hoptoad error
    flash[:warning] = SMTP_SERVER_ERROR_FLASH
    render :action => 'new'
  end
end

If you use Suspenders you’ll be pleased to find that we’ve included config/initializers/errors.rb for you pre-populated with both SMTP and HTTP exceptions.