GIANT ROBOTS SMASHING INTO OTHER GIANT ROBOTS

Twitter Search (formerly Summize) is sweet. I use it every day to find first-person commentary on thoughtbot’s work, Boston sports, national politics, and anything else that catches my fancy.

The Twitter Search API is also sweet. I wanted to use it a while back for Politweets. In combination with a simple cron job, the effect is commentary on the U.S. Presidential Election from around the world, seen within minutes of being tweeted.

Dustin Sallings wrote a great Ruby wrapper for the Summize API that he put on github. I forked it and added a shoulda test suite and documentation.

Introducing the Twitter Search gem.

Usage

Install the gem.

sudo gem install dancroak-twitter-search -s http://gems.github.com

Require the gem.

require 'twitter_search'

Set up a TwitterSearch::Client. Name your client (a.k.a. ‘user agent’) to something meaningful, such as your app’s name. This helps Twitter Search answer any questions about your use of the API.

@client = TwitterSearch::Client.new 'politweets'

Request tweets by calling the query method of your client.

@tweets = @client.query :q => 'twitter search'

Search Operators

The following operator examples find tweets…

• :q => ‘twitter search’ – containing both “twitter” and “search”. This is the default operator.
• :q => ‘“happy hour“‘ – containing the exact phrase “happy hour”.
• :q => ‘obama OR hillary’ – containing either “obama” or “hillary” (or both).
• :q => ‘#haiku’ – containing the hashtag “haiku”.
• :q => ‘from:alexiskold’ – sent from person “alexiskold”.
• :q => ‘to:techcrunch’ – sent to person “techcrunch”.
• :q => ‘@mashable’ – referencing person “mashable”.
• :q => ‘superhero since:2008-05-01’ – containing “superhero” and sent since date “2008-05-01” (year-month-day).
• :q => ‘ftw until:2008-05-03’ – containing “ftw” and sent up to date “2008-05-03”.
• :q => ‘movie -scary :)‘ – containing “movie”, but not “scary”, and with a positive attitude.
• :q => ‘flight :(‘ – containing “flight” and with a negative attitude.
• :q => ‘traffic ?‘ – containing “traffic” and asking a question.
• :q => ‘hilarious filter:links‘ – containing “hilarious” and linking to URLs.

Foreign Languages

The Twitter Search API supports foreign languages, accessible via the :lang key. Use the ISO 639-1 codes as the value:

@tweets = @client.query :q => 'programmé', :lang => 'fr'

Pagination

Alter the number of Tweets returned per page with the :rpp key. Stick with 10, 15, 20, 25, 30, or 50.

@tweets = @client.query :q => 'Boston Celtics', :rpp => '30'

Gotchas

• Searching for a positive attitude :) returns tweets containing the text :), =), :D, and :-)
• Searches are case-insenstive.
• The “near” operator available in the Twitter Search web interface is not available via the API. You must geocode before making your Twitter Search API call, and use the :geocode key in your request using the pattern lat,lngmi or lat,lngkm:

@tweets = @client.query :q => 'Pearl Jam', :geocode => '43.4411,-70.9846mi'

Usage with ActiveRecord and cron

You can get fancier with your setup, using queuing or another approach, but here’s a simple example using cron.

Schema:

create_table "tweets", :force => true do |t|
  t.string   "user_name",          :limit => 20,  :default => "", :null => false
  t.string   "body",               :limit => 140, :default => "", :null => false
  t.datetime "created_at",                                        :null => false
  t.datetime "updated_at",                                        :null => false
  t.integer  "twitter_id",         :limit => 11
end

add_index "tweets", ["created_at"], :name => "index_tweets_on_created_at"
add_index "tweets", ["twitter_id"], :name => "index_tweets_on_twitter_id"

Hit the API in your rake task (write the code in a class so it can be easily tested):

class Twitter
  def self.import!
    # the implementation is up to you
  end
end

Since I don’t have the cron API memorized, I like to create cron jobs with a handy little tool called crondle:

require 'lib/crondle'

Crondle.define_jobs do |builder|
  rails_root = '/var/www/apps/politweets/current'
  [3, 9, 15, 21, 27, 33, 39, 45, 51, 57].each do |minute|
    builder.desc "Import tweets at #{minute} past the hour"
    builder.job "#{rails_root}/script/runner Twitter.import!", :minute => minute
  end
end

Run the crondle script to get the text that you’ll put in crontab -e:

# Import tweets at 3 past the hour
3 * * * * /var/www/apps/politweets/current/script/runner Twitter.import!

# Import tweets at 9 past the hour
9 * * * * /var/www/apps/politweets/current/script/runner Twitter.import!

...

And you’re on your way.

Thanks to Doug, Gabe, Jason, Min, and Dustin for their work on Politweets and the Twitter Search gem.