Sailing down the Hudson with RVM

Nick Quaranto

We recently decided our CI server needed an overhaul. I really enjoyed Integrity as a build server, but after trying out Hudson it’s hard to say I want to go back. Hudson has several huge advantages:

  • Totally self contained. Integrity was a constant mashup. I don’t want to maintain the codebase for my CI server anymore.
  • Live console output. When your builds take 20-30 minutes, this is really nice.
  • Metrics/graphing built in! None so far on the other side of homegrown Ruby CI servers.
  • Kickass plugin installing, don’t need to hack a Gemfile or fix bad code (so far).
  • Building multiple branches in the same project is no longer a hassle.
  • Jobs won’t stomp over each other, and I don’t need separate configuration/database for them.

Here’s the details of what I went through to get our new CI server up and running. Be warned, these are probably specific to our projects and needs, but the general process can definitely help anyone who’s looking to get started.

Basic Loadout

We started with a fresh Ubuntu 9.04 LTS install. First up was a basic iptables setup, all the commands you need are on this SliceHost article.

Next up we needed to install Hudson. Follow the instructions here: http://hudson-ci.org/debian/

The nice part about this install (instead of custom compiling it) is that it comes with an init script that is really handy to stop/start the server. There’s more tips on installing here.

If apt-get starts whining, just run sudo apt-get install -f. The hudson package should install the Java dependencies you’ll need for now.

Let’s also install some other basics we’ll need eventually. You may disagree with my choices here, but too bad, I’m the one maintaining CI:

sudo apt-get install zsh vim git-core build-essential postfix ack-grep
curl
chsh /usr/bin/zsh

The postfix install will prompt you for your server’s host name, we’ll want it eventually for emails. If you’re confused, check this article out.

By now you should be able to visit http://YOURSERVERIP:8080 and Hudson should be alive and kicking.

Apache Setup

That’s annoying though, we want to serve requests on port 80. Supposedly doing this with Hudson itself is a bad idea, since it would need to run as root. Apache does a better job, and this tutorial goes over the reasons why.

Here’s what I had to do to get this working:

sudo apt-get install apache2 libapache2-mod-proxy-html
sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2dissite 000-default

That will install apache2, get the proxy mod enabled to forward requests along to hudson from the root domain, and disable the default site. Next step is editing /etc/apache2/apache2.conf and adding the following directive to the bottom:

# where your server lives!
ServerName ci.thoughtbot.com

Next up we’ll need to actually make a virtual host for apache to serve (thanks to this StackOverflow question for the right incantations). Create /etc/apache2/sites-available/hudson with the following:

<VirtualHost *:80>
  ProxyPass         /  http://localhost:8080/
  ProxyPassReverse  /  http://localhost:8080/
  ProxyRequests     Off

  # Local reverse proxy authorization override
  # Most unix distribution deny proxy by default
  # (ie /etc/apache2/mods-enabled/proxy.conf in Ubuntu)
  <Proxy http://localhost:8080/*>
    Order deny,allow
    Allow from all
  </Proxy>
</VirtualHost>

And the final touch to enable hudson and fire up apache:

sudo a2ensite hudson
sudo /etc/init.d/apache2 restart

Hudson Configuration

Port 80 is much better. Now to continue with the configuration! We’re just using a standard security setup with one user. The process is a bit confusing, you have to enable authorization, then go out and sign up to set the password. After that, make sure you set the anonymous user permissions so they can see jobs on the home page.

If you’re an idiot like me and locked yourself out while doing this, disabling security is pretty simple.

As for plugins, we chose so far:

  • Git SCM (allows us to check out from git)
  • GitHub (links files in builds on github)
  • Campfire (update: the github version has been merged in, install it from the plugin directory like normal)

After installing the plugins and restarting, I disabled a bunch we didn’t need too, like Maven, CVS, etc. I would also check the “Prevent CSRF” button, that’s just a ridiculous option not to have on.

Databases

Let’s get mysql and postgres installed first.

sudo apt-get install mysql-server mysql-client libmysqlclient15-dev
sudo apt-get install postgresql postgresql-client postgresql-contrib

Redis is up next. We’re running the release candidate of 2.0, which is a bit of a pain to set up so far since make install was removed. I came up with an upstart scriptto run it instead.

sudo adduser redis
sudo su - redis
curl -O http://redis.googlecode.com/files/redis-2.0.0-rc1.tar.gz
tar zxvf redis-2.0.0-rc1.tar.gz
cd redis-2.0.0-rc1.tar.gz
make
logout
sudo bash -c "curl http://gist.github.com/raw/444033/9784f426402d5bece09894fcc6f98a33db5f5d15/redis.upstart > /etc/event.d/redis"
sudo start redis

Finally, Ruby

Now that RVM exists, we can completely isolate projects from each other using gemsets. First we’re going to need even more system packages to deal with ruby and the other gems we’ll be installing eventually (nokogiri, paperclip, etc):

sudo apt-get install bison openssl libreadline5 libreadline-dev zlib1g zlib1g-dev libssl-dev libsqlite3-0 libsqlite3-dev sqlite3 libreadline5-dev libreadline6-dev libxml2-dev subversion autoconf libxslt-dev imagemagick

Jump in as the hudson user now and set up RVM:

sudo su - hudson
bash < <( curl http://rvm.beginrescueend.com/releases/rvm-install-head )
echo "[[ -s $HOME/.rvm/scripts/rvm ]] && source $HOME/.rvm/scripts/rvm" > .profile
echo "---\ngem: --no-ri --no-rdoc" > .gemrc

Install the rubies you need now. Our starting lineup:

rvm install 1.8.6
rvm install 1.8.7
rvm install ree
rvm install jruby

Maybe not finally

We’ll need to get the hudson user set up with git and github too. Make sure to upload your public key ~/.ssh/id_rsa.pub to github so you can start cloning repos.

ssh-keygen -t rsa
git config --global user.email "hudson@thoughtbot.com"
git config --global user.name "hudson"

For each app, create the gemset, databases, and get the gems you need. Bundler would make this easier, but we’re not on it for all projects yet.

sudo su - hudson
rvm gemset create hoptoad
rvm gemset use hoptoad
mysql -u root
> create database hoptoad_development;
> create database hoptoad_test;
gem install mysql nokogiri json

Make a new Hudson job, check from SCM periodically, enable the post build notifications you want. Here’s the build script we’re using for hoptoad so far:

bash -l -c "rvm use 1.8.6-p287 && rvm gemset use hoptoad && rake db:migrate db:test:prepare default"

Debug, rinse, and repeat until it’s green.

Made it

RVM made this process a lot easier, I used to have to compile multiple versions of Ruby on my own and it was just a pain to manage. Gemsets are a killer feature too, and it’s easy to move them around to another Ruby version with the copy command:

rvm gemset copy 1.8.6@factory_bot 1.8.7@factory_bot

Overall the process was pretty smooth, and like any CI solution this took a while to set up and has its share of annoyances:

  • Campfire plugin was obnoxious to figure out. Not sure why they ship a broken one, but the custom one from GitHub works great. The fixed plugin has been merged in now!
  • Unfortunately, I couldn’t figure out how to make GitHub’s post-receive URLs trigger a build. It worked with a GET request, but POST requests seem to need some special authentication. Hudson has a cron-job style SCM polling, so that works instead.
  • Lots of clicking around to configure everything, my vim-senses are tingling for a command line interface.

I hope these guidelines help those new to continuous integration with Hudson or otherwise. Of course, there’s plenty of other options in the Ruby-verse, but hopefully you won’t pass up what Hudson has to offer.

''