cpytel

The Road to Rails 3

Being at RailsConf 5 has given us the opportunity to finalize a lot of the work we’ve done to prepare our plugins and gems for Rails 3. Thankfully, for many of the most popular gems, we’ve been able to maintain both Rails 3 and Rails 2.3.x compatibility in one gem. However, we’re taking this opportunity to say goodbye to some of our less widely used plugins, and some we plan on dropping Rails 2 support for altogether.

Obviously, Rails 3 isn’t actually out yet, so what we’re talking about here is Rails 3 beta 4. We’ll continue to keep things up to date and tested as we all move toward the release of Rails 3. Your help and patches are more than welcome.

So here is a comprehensive overview of the current status of the projects for both Rails 3 beta 4 and Rails 2.

Paperclip

We released Paperclip 2.3.3 a few days ago. This new version of Paperclip will work with Rails 3. Thanks to the investigation of nragaz and help from isaac and joeljunstrom on github, we worked out the kinks and it should be working with the Rails 2.3.x line, and Rails 3-beta 4. For the latest version of Paperclip, we’re no longer officially supporting Rails 2.0.x. The earliest version that will work is Rails 2.1.0. If you need support for an older version of Rails than that, you can use Paperclip 2.3.1.1.

hoptoad_notifier

A few days ago we released hoptoad_notifier 2.2.6 with includes support for Rails 3-beta 4 as well as all versions of Rails 2.x and Rails 1.2.6.

shoulda

We just released shoulda 2.11. Along with Rails 3 support, we’re maintaining support for Rails 2.3.x in this latest release. However, the latest version of shoulda will not support versions of Rails less than 2.3. If you need support for a version of Rails older than that, you can use a previously released version.

In addition to the Rails 3 support, shoulda 2.11 introduces some dramatic changes to shoulda, including a new way of interacting with all shoulda macros. The previous way has been deprecated and will be removed in shoulda 3.0. We’ll make a separate blog post detailing many of the very cool changes to shoulda and more details about the future of shoulda soon, but for now, take a look at the README for the latest information on setting up and using shoulda.

Factory Girl

We just pushed factory_girl 1.3 and factory_girl_rails 1.0. This new version adds Rails 3 support. Because of the way that Rails 3 loading has changed, we’ve decided to make a separate factory_girl_rails gem that will be used for when you want to use factory_girl with Rails. The existing factory_girl gem is used by factory_girl_rails and would be used if you’re using factory_girl outside of Rails. If you want to use factory_girl with Rails 2 you can continue to use the base factory_girl gem.

Clearance

We just released Clearance 0.9.0.rc1. This is a release candidate for Clearance 0.9.0. This new version adds support for Rails 3 but drops support for Rails 2. Don’t fret, if you won’t be upgrading to Rails 3, you can use a previously released version of the gem (0.8.8). We’re doing this one as a release candidate because of the dropping of backwards compatibility and the fact that we haven’t had a chance to test the new version in a variety of Rails 3 apps using clearance.

Please flex this release candidate with your Rails 3 apps and let us know how it goes.

Suspenders

Suspenders is currently at 2.3.5 (we haven’t been able to upgrade to 2.3.8 because of bugs we’ve seen with mongrel, webrat, and rack). We anticipate that Suspenders will be upgraded to Rails 3 a little after Rails 3 final comes out. But to be honest, we’re actually not sure yet what the upgrade path will look like for applications that are currently tracking Suspenders. It may be impossible to do without so many conflicts that its not worthwhile. We’re going to have to work on this more and keep you posted. Additionally, we’re in the process of making some fairly dramatic changes to Suspenders. Watch it on github and stay tuned here for more.

High Voltage

Fire in the Disco! We’ve also released High Voltage 0.9.0 which supports Rails 3 and is now a gem (it was previously just a plugin). The new version also drops support for Rails 2. If you need the previous, Rails 2 plugin there is a rails2 branch you can retrieve it from.

Pacecar

We also just released Pacecar 1.3 which supports Rails 3 and drops support for Rails 2. As in the other cases where we’ve done this, you can use the previous version of the gem, version 1.2.0 with Rails 2, or track the rails2 branch.

Squirrel

Squirrel was born out of a desire to make a new query syntax that was dynamic while being clean and simple. With Rails 3’s introduction of the New Active Record chainable query language, that goal has now been achieved in Rails. As a result, we’ll no longer be maintaining Squirrel. It was a fun ride.

Mile Marker

Over time, our workflow slightly changed for how we built applications and we haven’t used Mile Marker ourselves for some time now. As a result, we’re taking this opportunity to cease maintenance of this plugin and bid it farewell.

Moving on down the road

We’ve gotten more and more familiar with Rails 3 during moving all these gems to it. Many of the new features it offers are great, and existing features have been improved and cleaned up. We’re looking forward to Rails 3 finally being released in the coming weeks. Now that our plugins are up and running it should help us all to transition smoothly and quickly.

Thanks to the core team and various other railsconf attendees for spending time with us this week working on some of this - we’re looking forward to the final version of rails3!

August 25, 2009

dancroak

Testing Paperclip on S3 with Cucumber & Factory Girl

We’ve been using Heroku as a staging environment for our latest project. One constraint is a read-only filesystem.

The most apparent effect of this is we cannot allow users to upload files to the filesystem.

Fine. Paperclip has an S3 storage option.

Testing

Webrat has a very nice existing convention for interacting with file fields:

When /^I attach the file at "([^\"]*)" to "([^\"]*)"$/ do |path, field|
  attach_file(field, path)
end

Unfortunately, if we have an S3-backed model like this…

has_attached_file :logo,
 :path           => ":attachment/:id/:style.:extension",
 :storage        => :s3,
 :s3_credentials => {
   :access_key_id     => ENV['S3_KEY'],
   :secret_access_key => ENV['S3_SECRET']
 },
 :bucket         => ENV['S3_BUCKET']

… then we’re going to be doing a RESTful PUT to S3 during each test run.

Incidentally, those ENV variables are Heroku’s config vars. The idea is that you keep that configuration separated from your source control.

Total integration?

You could argue that these PUTs to S3 are a good thing because your Cucumber feature will represent total integration.

While that’s true, I’d rather not pay bandwidth costs and I’m comfortable as long as the correct interface to S3 was called.

So what we’ve landed on is something like this:

Given I am on the new band page
When I attach a "demo_tape" "mp3" file to a "band" on S3
And I press "Upload demo tape"
Then I should see "Band was successfully created"

The only non-standard Webrat step is our new S3 step. Let’s take a look at it:

# features/step_definitions/paperclip_steps.rb

When /^I attach an? "([^\"]*)" "([^\"]*)" file to an? "([^\"]*)" on S3$/ do |attachment, extension, model|
  stub_paperclip_s3(model, attachment, extension)
  attach_file attachment,
              "features/support/paperclip/#{model.gsub(" ", "_").underscore}/#{attachment}.#{extension}"
end

The stub_paperclip_s3 method is coming from a custom Shoulda Macro:

# test/shoulda_macros/paperclip.rb

module Paperclip
  module Shoulda
    def stub_paperclip_s3(model, attachment, extension)
      definition = model.gsub(" ", "_").classify.constantize.
                         attachment_definitions[attachment.to_sym]

      path = "http://s3.amazonaws.com/:id/#{definition[:path]}"
      path.gsub!(/:([^\/\.]+)/) do |match|
        "([^\/\.]+)"
      end

      FakeWeb.register_uri(:put, Regexp.new(path), :body => "OK")
    end

    def paperclip_fixture(model, attachment, extension)
      stub_paperclip_s3(model, attachment, extension)
      base_path = File.join(File.dirname(__FILE__), "..", "..",
                            "features", "support", "paperclip")
      File.new(File.join(base_path, model, "#{attachment}.#{extension}"))
    end
  end
end

class ActionController::Integration::Session
  include Paperclip::Shoulda
end

class Factory
  include Paperclip::Shoulda
end

Fakeweb and more conventions

We’re using the Fakeweb gem like we normally use mocking: expect that something happened, and stop it from actually happening.

We’re also leaning on conventions similar to the actor directory convention we’re also trying.

In this case, we’re expecting our features directory to look like this:

features/support/paperclip/band/demo_tape.mp3
features/support/paperclip/band/demo_tape.aac
features/support/paperclip/band/demo_tape.ogg
features/support/paperclip/band/demo_tape.wav
features/support/paperclip/user/avatar.png
features/support/paperclip/user/avatar.jpg
features/support/paperclip/user/avatar.gif

This allows us to test the expected and unexpected formats by changing this line:

When I attach a "demo_tape" "mp3" file to a "band" on S3

Factories

The reason the stub_paperclip_s3 and paperclip_fixture methods are set up as a custom shoulda macro is so that you can use them in your factory code:

Factory.define :band_with_demo_tape, :parent => :band do |band|
  band.demo_tape { band.paperclip_fixture("band", "demo_tape", "png") }
end

Go or no go?

It’s been quite useful so far for us. How are you testing Paperclip uploads to S3? Do you see any way to improve this step definition or the convention?

May 15, 2008

jyurek

Paperclip Tips and Updates

So there’s been a bit of activity on the Paperclip front. I’ve added Amazon S3 support using the RightAWS gem. Some fellow githubbers have contributed some patches to fix up said S3 support, as well as to add more (and better) validations for content type, etc. We’ve brought it up to a nice, round v2.1.2 as of yesterday. Give yourselves a round of applause.

I highly encourage you to give it a whirl if you haven’t already. It’s very easy to both get and to use!

An interesting note!

Ken Robertson has gone to great lengths to port Paperclip to DataMapper, which should be great news for all you Merbers out there. He’s kept it quite up-to-date, and it’s right alongside the 2.1.2 current release. Hopefully we’ll be able to merge codebases in the future, though right now there’s enough of a difference between DM and AR to make that not terribly feasible.

Here’s a handy tip!

One of the most frequently asked questions is how to use data from your instances in the path and/or URL. The answer is the interpolations hash, which is quite user-extensible. Let’s say you had a song that played in the background of each User’s profile (making this exercise purely hypothetical, of course), and you wanted its name to be the same as the User’s username.

One really good way of allowing this functionality would be to add the following to your config/initializers/paperclip.rb file:

Paperclip::Attachment.interpolations[:username] = proc do |attachment, style|
  attachment.instance.login # or whatever you've named your User's login/username/etc. attribute
end

The #instance method is the instance of the model that this attachment is attached to. You can access any of the model’s attributes, methods, associations, and so on from that object just like normal. Also note that you don’t have to name your interpolation the same thing as the attribute you’re interpolating (though it helps clarity).

Now you can add this to your :path and :url parameters like so:

class User < ActiveRecord::Base
  has_attached_file :song,
                    :path => ":rails_root/public/system/:attachment/:username.:extension",
                    :url  => "/:attachment/:username.:extension"
end

When you call #url or #path on your attachment, Paperclip will run through the interpolations hash, find strings that match its keys, and replace them with the return values of the procs. In this case, you’d produce a lovely url of ”/songs/jyurek.mp3”

It’s a very easy way to add flexibility to your files’ names without having to modify any code yourself.

Another handy tip!

S3 support is now baked in! The updated has_attached_file call looks kinda like this one does:

class House < ActiveRecord::Base
  has_attached_file :blueprint,
                    :styles => { :thumbnail => "150x150" },
                    :path => ":attachment/:id/:style.:extension",
                    :storage => :s3,
                    :s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
                    :bucket => "bucket-of-holding"
end

Please note the additional options, including:

:storage: Specify :s3 here to use S3. Strictly speaking you could specify :filesystem here if you’re using the filesystem, but it’s the default so don’t bother.
:s3_credentials: You should give this a Hash, a path to a file, or a File object. The File should contain a YAML-ized Hash, and the contents of the Hash should be the access_key_id and secret_access_key used to access your S3 account. You can also “environment-space” these inside the hash, just like your database.yml file.
:bucket: The name of the S3 bucket that will be holding your data.

Note that there’s no :url option. That’s because the S3 URLs are generated from the bucket and the path name, so you don’t have to worry about them. You can also specify permissions for your files, in case you don’t want the default “public-read”, by using :s3_permissions.

You can find more about the S3 Storage options and the Filesystem Storage options at their RDocs.

Keep up to date!

Remember, there’s always the Paperclip Google Group and the Paperclip Lighthouse Account in case you have problems, questions, or feature requests.

April 15, 2008

jyurek

Moving to Github

We’re moving our plugins, gems, and other open source projects to github. I know, it’s a shock. We’re going to keep the SVN versions of them going for a while, but the “official” repositories for each of these is now what’s hosted on github. So get busy with the hot forking action.

We’ve got:

By-and-large, we’ve resisted the move to git, despite the searing heat it’s been putting off in the Ruby community recently. We’re still a centralized development house, and so our internal development will still be SVN-based. It’s just how we roll. Though I imagine a few of the guys here will start picking up git-svn.

March 18, 2008

jyurek

For attaching files, use Paperclip

Last Update: For archival reasons, I’m removing direct links to releases, etc. from here. All the info you need is at the official paperclip project page.

For some reason, file attachment is annoying. I don’t know why, and I know a lot of people have attempted to solve the problem in the past, myself included. Yet it still is. Having gotten fed up with gotchas and design decisions that we didn’t agree with, I went and wrote Paperclip on the plane to RailsConf last year. We’ve been using it here in various forms since and IMHO it’s the way to handle uploads, and finally decided that it should be released.

class User < ActiveRecord::Base
  has_attached_file :avatar,
                    :styles => { :square => ["64x64#", :png],
                                 :small  => "150x150>" }
end

A file is treated like any other attribute. It’s assigned like any other attribute, and it’s not saved until you call #save. It doesn’t have its own model. You can say where it’s saved on the filesystem, and what URL it’s referred to by (which means you can let Apache/nginx handle it or you can route it through the app for permissions/security). You can say what thumbnails are made, what resolution and format they are, and you can actually save cropped square thumbnails without any hassle.

class AddAvatarToUser < ActiveRecord::Migration
  def self.up
    add_column :users, :avatar_file_name, :string
    add_column :users, :avatar_content_type, :string
    add_column :users, :avatar_file_size, :integer
  end

  def self.down; ...; end
end

You don’t need mini_magick (which we’ve found may have been causing issues), you don’t need ImageScience (which, on the first image we tried to upload, failed to decode it, and it was a simple GIF), you don’t need RMagick (which has memory issues that were the impetus for mini_magick in the first place). You just need ImageMagick installed somewhere, which is as easy as yum, apt-get, or port on any system worth hosting on.

And this isn’t just for avatars and images. You can upload anything. No thumbnails are made by default, so it won’t automagically choke on your Excel docs.

Usage

In your model:

class User < ActiveRecord::Base
  has_attached_file :avatar, 
                    :styles => { :medium => "300x300>",
                                 :thumb  => "100x100>" }
end

In your edit and new views:

<% form_for :user, :html => { :multipart => true } do |form| %>
  <%= form.file_field :avatar %>
<% end %>

In your controller:

def create
  @user = User.create( params[:user] )
end

In your show view:

<%= image_tag @user.avatar.url %>
<%= image_tag @user.avatar.url(:medium) %>
<%= image_tag @user.avatar.url(:thumb) %>

Installation

piston import https://svn.thoughtbot.com/plugins/paperclip/trunk

It can’t be everything for everyone, but for the vast majority of cases we’ve come across, this is the right fit. Really, once you use this you’ll wonder why managing files was such a hassle.

giant robots smashing into other giant robots

Written by thoughtbot

The Road to Rails 3

Paperclip

hoptoad_notifier

shoulda

Factory Girl

Clearance

Suspenders

High Voltage

Pacecar

Squirrel

Mile Marker

Moving on down the road

Testing Paperclip on S3 with Cucumber & Factory Girl

Testing

Total integration?

Fakeweb and more conventions

Factories

Go or no go?

Paperclip Tips and Updates

Moving to Github

For attaching files, use Paperclip

Usage

Installation