A Healthy Bundle

If you’re writing Rails applications today, then you’re probably enjoying Bundler when adding dependencies to your Rails applications. Bundler lets you declare your application’s dependencies clearly and concisely, and it will figure out which available versions satisfy your requirements.

Version Requirements

Assuming that your dependencies are released gems, there are three ways to specify your dependencies:

# exact version
gem 'nokogiri', '1.0.3'
gem 'webrat', '0.3.1'

# pessimistic version
gem 'nokogiri', '~> 1.0.3'
gem 'webrat', '~> 0.3.1'

# any version
gem 'nokogiri'
gem 'webrat'

Let’s take a look at some of the ups and downs of each approach.

Exact Version

If you specify the exact version of every gem, then you’ll never have unexpected breaks while upgrading. Running bundle is the same as running bundle update, and you’ll always know exactly what’s installed. You won’t have to worry about updating capybara and getting a new version of rack that may affect your production environment.

It has some major downsides, though. You can only keep a tight lock on every gem if you also specify the exact version of every dependency, including your dependencies’ dependencies, and so on. This means that you’ll need to chase down a viable version of every gem in use in your application’s bundle, even if you’re not using it directly. It also means that you need to track when grand-dependencies change or are no longer required.

It means that Bundler can no longer do its job well. Every version you specify removes options from Bundler’s resolution algorithm, which means you have to do all the work yourself. Let’s look at a simple upgrade scenario: let’s say you decide to upgrade from webrat 0.3.1 to webrat 0.4.0. You change your Gemfile:

gem 'nokogiri', '1.0.3'
gem 'webrat', '0.4.0'

And run bundle to install the new version.

However, this particular version of webrat depends on a newer version of nokogiri, so you’ll have to bump nokogiri gem as well. If you have other dependencies pegged to a specific version of nokogiri, you’re now privileged to play a fun game of dependency whack-a-mole. Common gems like rack and json are particularly problematic in this respect.

In summary, specifying exact versions gives you a lot of control and insight into dependencies in your applications, but it makes adding and updating dependencies difficult.

Pessimistic Version

Specifying a pessimistic version tells Bundler that it’s fine to upgrade a gem, as long as the gem version doesn’t change too much. This works best in conjunction with gems which follow semantic versioning. With semantic versioning, you can tell Bundler to upgrade gems as long as the new versions only contain bug fixes by pegging a minor version. You can give Bundler more freedom by pegging a minor version, which will mean that only backwards-compatible changes are allowed.

Loose versions also have drawbacks, however. Although semantic versioning says that gems shouldn’t break compatibility, there’s no guarantee that somebody won’t do so by mistake, or introduce an unrelated bug. You also can’t depend on every gem following semantic versioning.

Looser versions also increase the number of potential changes which occur when you change your Gemfile, which makes it more likely that you’ll accidentally break something when adding a new dependency.

By specifying pessimistic versions, you give up some of the control from exact versions, but adding and updating dependencies becomes much easier, and versions won’t change to drastically without your knowledge.

Versionless

You can also tell Bundler that you don’t care at all by leaving the version out entirely. In this case, Bundler will install the latest versions of all your dependencies that work together. It’s worth noting that you still get a reliable development environment, because Bundler stores the versions of all your dependencies separately in Gemfile.lock. This means that running bundle won’t change the installed versions of any dependencies it doesn’t need to.

Leaving the version out is less work initially, since you don’t have to look up the version or add it to the file. It doesn’t restrict Bundler at all, which means it can do the best job possible of finding versions that work together. It also makes it extremely easy to update all your gems, since everything will get the latest version from a bundle update.

However, without placing any restrictions on the version, you open yourself up to dramatic updates in your gems. Running a bundle update may result in backwards-incompatible changes to your dependencies, and the risk of introducing newer versions of gems with new bugs is higher.

Be Purposeful

The benefits and drawbacks to these three approaches have led us to a combined approach that treats each gem specially.

Frameworks like Rails are especially likely to introduce breaks, even with patch updates, so locking such gems to an exact version is best. Upgrading your framework needs to be handled manually as a separate task, and locking to an exact version allows you to decide when to undertake that work.

Many gems have a strong history of backwards-compatibility, such as pg, thin, and debugger. Breakage from upgrading those gems is unlikely and will be spotted immediately, so specifying a version number is unnecessary.

Other gems frequently have backwards-incompatible changes but are reliable between minor or patch versions. Examples include rspec, factory_girl, and capybara. These gems are good candidates for pessimistic versions.

Using the correct version restriction for each gem expresses purpose and intention for those gems. Using an exact version represents a fragile dependency; using no version represents a safe and loose dependency that can be updated often. When updating or adding dependencies, use an intention-revealing git commit message that delivers meaningful information to the current and future team.

Take Little Steps

You’re going to need to upgrade your application’s dependencies. Security vulnerabilities are discovered, new features arrive to increase productivity, and bugs are fixed. Interdependencies will force you to update more gems than you intend to, and each upgrade requires debugging and fixing incompatibilities in your application. Therefore, it’s best to upgrade continually, in little steps.

If you tighten the reins too much on your dependencies, it will discourage you from updating frequently. Find a balance that works for your application, your team, and your dependencies, but make sure that you give yourself enough room for continuous updates.

We’d like to thank Josh Susser for contributing to this discussion and helping us to find our healthy mix.

What would happen if you ran bundle update right now?

Is there a bundle command to tell me what would be updated with bundle update, without actually making those updates?

As it turns out there is! Bundler 1.1 introduces a new command:

bundle outdated
    Show all of the outdated gems in the current bundle.  
    This will give you a report of gems that have newer versions available.

By itself, this will list all of the gems in your Gemfile.lock that have newer versions, and what the current and latest versions are.

This gives a good preview of what you are up against if you want to get your gems up to date. From there, use git commits to make incremental changes.

Putting it to use

Today I created an app with the suspenders gem, and noticed it is running on Rails 3.1.1, which I wanted to upgrade to 3.2.2.

I also saw a few other gems with version attributes:

gem 'sass-rails',   '~> 3.1.4'
gem 'coffee-rails', '~> 3.1.1'
gem 'uglifier', '>= 1.0.3'
gem 'rspec-rails', '~> 2.6.1'
gem 'cucumber-rails', '1.1.0'
gem 'capybara-webkit', '~> 0.7.1'

First I ran rake to confirm all tests were passing, bundle update to make sure the gems were all up to date with the versions specified, and created an initial commit.

Next I upgraded to rails 3.2.2 and reran bundle update, and found out (after a couple itterations) that sass-rails and coffee-rails had to be updated too, because of there dependance on ActiveSomethingOrOther. Updating these three gems to their latest versions allowed bundle update to do it’s thing.

I reran my tests, and did ran git diff Gemfile.lock to see exactly what was new. For giggles I reran bundle outdated and rejoiced at the shrinking list. Time to commit with a message about the gem version changes.

From here it was rinse and repeat, checking rubygems.org and looking at the dependencies, making small changes, bundling, raking, committing.

At the end of the road I had a good series of commits spelling out exactly what was needed to get to my goal of a fully updated rails 3.2.2 environment.

If you run bundleoutdated and all you see is:

Outdated gems included in the bundle:  
* sprockets (2.4.0 > 2.1.2)  

Then you are doing it right!

Have a cookie!

Read more about what’s new with bundler 1.1 from Pat Shaughnessy.

Use Bundler’s binstubs!

If you’re not using bundler’s binstubs with RVM integration yet, you should give it a try! This means you don’t have to type “bundle exec” ever again.

Setup:

• One time, run chmod +x $rvm_path/hooks/after_cd_bundler
• Once for each project, run bundle install -—binstubs

With rvm integration enabled, the “bin” directory is added to your path each time you cd into a project directory with binstubs. That means you can just run “rake”. If you aren’t ignoring the “bin” directory in your project, you should do so:

# .gitignore
bin/

Appraisal: find out what your gems are worth

Since the introduction of bundler to the Ruby community, dealing with dependencies has gotten much easier. Almost every library now has a Gemfile that looks like this:

source "http://rubygems.org"
gemspec

This pulls runtime and development dependencies from your project’s gemspec and finds versions that can all agree with each other. Requiring any of your dependencies will always get the expected version regardless of order, and updating dependencies can be performed with a single command.

However, there’s still one piece missing from this puzzle: how do you make sure that your library works with all supported versions of your dependencies? A common example is Rails: now that Rails 3.1 is out, how can you make sure that your library works with both Rails 3.0.x and Rails 3.1.x?

One side effect of using bundler is that all dependencies are “locked.” This essentially means that, whether you’re running your tests on your home computer, work computer, or continuous integration server, your dependencies are always the same exact versions. This obviously means that you’ll only ever test against one version of Rails; Yehuda Katz, one of the authors of bundler, has a partial answer to this problem:

“When developing a gem, use the gemspec method in your Gemfile to avoid duplication. In general, a gem’s Gemfile should contain the Rubygems source and a single gemspec line. Do not check your Gemfile.lock into version control, since it enforces precision that does not exist in the gem command, which is used to install gems in practice. Even if the precision could be enforced, you wouldn’t want it, since it would prevent people from using your library with versions of its dependencies that are different from the ones you used to develop the gem.”

Unfortunately, there are some holes in this practice.

For one, this means that tests that pass on one machine may not pass on another, even if the library code hasn’t changed at all. This can be frustrating, as it breaks the general expectation that a fresh checkout of any master branch should have passing tests.

For another, this doesn’t reliably exercise your library across versions of your dependencies; rather, it hopes that you happen to run the tests on enough varied environments between each release that you happen to catch any regressions.

If locking your dependencies hides errors, and not locking your dependencies creates them, how can you reliably run your tests on different versions of your dependencies without creating an unstable development environment? The answer is appraisal.

Appraisal runs your tests across configurable, reproducible scenarios that describe variations in dependencies. As an example, if you need to test compatibility with several versions of Rails, you can use an Appraisals file like this:

appraise "3.0" do
  gem "rails", "~> 3.0.11"
end

appraise "3.1" do
  gem "rails", "~> 3.1.1"
end

You can include as many appraise blocks as you want, and you can be as specific or loose with requirements as you like. Running rake appraisal:install will generate separate bundler-compatible Gemfiles for each scenario, combining the base requirements from your main Gemfile with the changes in each scenario. You can run any rake task across every scenario using the “appraisal” task:

# Run your specs on Rails 3.0 and 3.1
rake appraisal spec

When using Appraisal, we recommend that you check your Gemfile.lock into version control, as well as the gemfiles directory generated by Appraisal. This will ensure that your tests are always green when you check out a fresh copy. It also allows you to create reproducible regression tests for version-specific bugs.

We’ve successfully used Appraisal on numerous projects this year. If you’d like to see real world examples, read the Appraisals files for Clearance, Factory Girl, Capybara Webkit, Copycopter Client, and Paperclip.

Do you have a gem that supports multiple version of Rails or any other library? Find out what your gem is worth: install appraisal.

2011 Rubyist’s guide to a Mac OS X development environment

It’s been two and a half years since my last laptop. It’s neat to look back and see how much has improved since then for setting up a Ruby development environment.

Of particular note, Homebrew, RVM, and Bundler did not exist back then.

Here’s how I set up an OS X 10.7 (Lion) thoughtbot laptop in 2011.

GCC

I need GCC to help install everything else so I downloaded GCC for Lion.

We used to have to install XCode to get GCC when OS X wasn’t for developers, which was a 3-4GB download and took 10GB+ of space. Buzzkill.

However, Kenneth Reitz, one of the Readability guys, fixed this with his OS X GCC installer, which is a comparatively svelte 272MB download.

Later on, when we’re installing things using Homebrew, we’ll see warnings like:

Xcode is not installed! Builds may fail!

But, the builds will build fine.

While that’s installing, we’ll customize our environment a little.

SSH

I need a public key to get access to private Github repositories.

ssh-keygen -t rsa

I’m kept hitting “enter” until it was done. Alternatively, I could have brought my old SSH key over but I’m not into falconry.

dotfiles

We have a standard set of configurations for vim, irb, git, zsh, and more.

I cloned the repo:

git clone git://github.com/thoughtbot/dotfiles.git

I ran the installer:

./install.sh

This sets up the appropriate symlinks (~/.vimrc, ~/.irbrc, etc.). I’ll stay up-to-date and contribute using the fork-track-update flow described in the README.

zsh

Our dotfiles assume zsh so I switched from the bash default to zsh:

chsh -s /bin/zsh

Re-map Caps Lock to Control

We’re pretty much all vim users here so it’s nice having super-quick home-row access to the Control key… and who uses Caps Lock, anyway?

System Preferences > Keyboard > Modifier Keys

git

Already installed by default, but I set the global config:

git config --global user.name "Your Name"
git config --global user.email you@example.com

Heroku accounts

I’m using Heroku for all my apps right now. However, thoughtbot’s clients and even our own apps like Trajectory are not owned by my Heroku account. So, it comes in handy to be able to switch to a different account on a project basis.

heroku plugins:install git://github.com/ddollar/heroku-accounts.git
heroku accounts:add dan  --auto
heroku accounts:add thoughtbot  --auto
heroku accounts:add client  --auto
heroku accounts:default dan

thoughtbot’s laptop script

Once GCC is downloaded and installed, I’m ready for the heavy-duty installation using our laptop script.

bash < <(curl -s https://raw.github.com/thoughtbot/laptop/master/mac)

This installs:

• Homebrew (for managing operating system libraries)
• Postgres (for storing relational data)
• Redis (for storing key-value data)
• Ack (for finding things in files)
• Tmux (for saving project state and switching between projects)
• ImageMagick (for cropping and resizing images)
• RVM (for managing versions of the Ruby programming language)
• Ruby 1.9.2 stable (for writing general-purpose code)
• Bundler gem (for managing Ruby libraries)
• Rails gem (for writing web applications)
• Heroku gem (for interacting with the Heroku API)
• Taps gem (for pushing and pulling SQL databases between environments)
• Postgres gem (for making Ruby talk to SQL databases)

It took about 15 minutes for everything to install.

While it’s running, it copies your SSH key to the clipboard and opens your Github SSH page. Paste your SSH key so your Github account is authenticated to your machine.

Caveats

We wrote a laptop script because we help hundreds of people a year get a Ruby development environment set up at workshops and Boston Ruby hackfests. One time we set up 30 business school students’ laptops in 3 hours.

If you read our source, you’ll see it it’s very simple but more invasive than, say, the excellent Cinderella by Corey Donohoe, which uses Chef to keep your machine tidy. We’re assuming the person definitely wants a “thoughtbot laptop”.

DIY

It’s pretty easy to write a wrapper that installs Homebrew, RVM, and your favorite databases and gems so consider forking our project and writing your own script, just like you might write your own Rails template script like Suspenders.