Get Your C On

This year, whyday happened to fall on the first day of Capeco.de. I’d been interested in playing with C for a while, so I decided to sit down and start learning. The K&R helped a lot, but digging through both Redis and Potion were a lot more insightful of how C could actually be used.

Since then, I’ve been trying to get some C in my Ruby by writing gems with C extensions. This isn’t hard to do, per se, but it was difficult to find a comprehensive list of requirements (as well as guidelines for organizing the code). I ended up looking at one of the most-used Ruby gems with C - Nokogiri.

Where to Start

You’ll want a way to test the code, as well as the rake-compiler gem. I don’t test my C explicitly - think of it as a handful of private methods. You’ll want to test the public methods on whatever classes and modules you write, so a C testing framework is overkill. Finally, you’ll want to become acquainted with ruby.h.

I decided to start simple and write a Sieve of Eratosthenes. I’d written one in pure Ruby and wanted a basic translation to C. I was also interested in benchmarking the code since I knew C would be a lot faster in this instance.

The sieve gem can be found here and its source here.

Directory Structure

A C extension’s directory structure is very similar to other Ruby gems; the only addition is an ext directory that will store files necessary for generating a Makefile and compiling the code. This is where your C files and their headers will go. In my case, these files are located in ext/sieve.

extconf.rb

extconf.rb is what will generate your Makefile. You’ll need to require "mkmf" and then call create_makefile("your_gem/your_gem"). The mkmf documentation is an excellent resource if you want to include other libraries or customize anything. My gem is straightforward so all I did was ensure the Makefile was created.

sieve.h

My sieve.h is very straightforward and doesn’t really need any explanation.

sieve.c

This is the meat and potatoes of the gem.

At the top of the file, you’ll need to #include <ruby.h> as well as any other headers you need.

You’ll also need an Init_your_gem() function that will be called similarly to main(). This is where I create the structure of my classes and modules for my sieve. I create a Sieve module, add a sieve instance method, and then have the Numeric class include Sieve.

Finally, there’s the sieve function itself. It returns a Ruby object which is of type VALUE. It also accepts a Ruby object (self), which is also of type VALUE. An important reminder is to make sure you free any memory you allocate or your gem will leak memory, just as you would in C.

The /lib directory

Although most of the actual work is done in C, we’ll want to have a bit of Ruby in the /lib directory. I’ve written a scaffold of the module at lib/sieve.rb, which has a require "sieve/sieve" at the top of the file (remember in extconf.rb when we passed a string to create_makefile? That’s it.).

The Rakefile

Being able to compile the gem and run the tests is important, which is why I mentioned the rake-compiler gem earlier. After requiring rubygems, rake, and your library, you’ll want to require "rake/extensiontask". That’ll give you a couple of handy rake tasks, namely clean and compile.

I like to set my default task to run tests, but you’ll want a couple prerequisites to that task: clean and compile. This will ensure that you’re rebuilding your gem and running with the latest compiled version.

Since I’m using Cucumber, it looks like this:

require "cucumber/rake/task"
Cucumber::Rake::Task.new(:cucumber => [:clean, :compile]) do |t|
  t.rcov = true
end

task :default => :cucumber

I also have my benchmark task here so I can find out how much more performant this library is compared to a pure Ruby implementation.

Testing

You’ll want to test your C extension just like any other Ruby gem. I prefer Cucumber but anything will do. I used a scenario outline for some of the basic primes and then found a file of the first one million primes for some heavy-duty lifting. I also tested that if enough memory couldn’t be allocated, it would raise a Ruby NoMemoryError exception.

Since you’re going to want to run the features against the latest changes of your gem (and not a version of the gem that’s installed), you’ll want to modify the load path within your test helper (test/test_helper.rb, spec/spec_helper.rb, or features/support/env.rb). My env.rb looks like this:

$LOAD_PATH.unshift(File.dirname(__FILE__) + '/../../lib')
require "sieve"
require "spec/expectations"

Building the gem

The gemspec for a Ruby C extension is fairly straightforward. The only thing you’ll need to add is to set the spec’s extensions attribute to the path to the extconf.rb file.

Gem::Specification.new do |s|
  s.require_paths = ["lib"]
  s.extensions = ["ext/sieve/extconf.rb"]
  # ... rest of the gemspec
end

As with any gemspec, you’ll want to make sure that you list the .c and .h files within files.

Results

Armed with this, you should be able to go and write Ruby C extensions to your hearts content. As for my Sieve experiment, here’s the pure-Ruby implementation of the sieve:

# usage:
#   >> sieve 100
#   => [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]
def sieve(n)
  numbers = (0..n).map {|i| i }
  numbers[0] = numbers[1] = nil
  numbers.each do |num|
    next unless num
    break if num**2 > n
    (num**2).step(n, num) {|idx| numbers[idx] = nil }
  end
  numbers.compact
end

My benchmarks were running the sieve on numbers from zero to one million in steps of 100,000. No memoization is used for either form.

On Ruby 1.8.7, here are the results from my benchmark:

                   user     system      total        real
sieve method   4.460000   0.060000   4.520000 (  4.522069)
Numeric#sieve  0.040000   0.000000   0.040000 (  0.046349)

Ruby 1.9.2 is significantly faster, but still doesn’t hold a candle to the C extension:

                   user     system      total        real
sieve method   2.410000   0.060000   2.470000 (  2.468430)
Numeric#sieve  0.050000   0.000000   0.050000 (  0.049053)

What I Learned

Writing C is both fun and can enhance performance of number-crunching and other fun things. It has it’s place and is a great addition to any Rubyist’s toolbox. Have you written any C extensions purely for performance gains? If you’re open to sharing the context, I’d love to hear about it!

MyGem.configure block

We recently made some enhancements to Clearance. One of those was to replace a manual process of the developer setting constants in their Rails environment files with a configuration block that could be used in config/initializers/clearance.rb.

I liked the way Hoptoad does it and wanted to implement the same pattern:

HoptoadNotifier.configure do |config|
  config.api_key = 'your_key_here'
end

So let’s implement:

module Clearance
  class << self
    attr_accessor :configuration
  end

  def self.configure
    self.configuration ||= Configuration.new
    yield(configuration)
  end

  class Configuration
    attr_accessor :mailer_sender

    def initialize
      @mailer_sender = 'donotreply@example.com'
    end
  end
end

We have a configure class method that stores a Configuration object inside the Clearance module.

Anything application developers can set from their configure block is an attr_accessor on the Configuration class.

side note - I used the name mailer_sender to match the Devise API.

So now, we can have a config/initializers/clearance.rb:

Clearance.configure do |config|
  config.mailer_sender = 'donotreply@example.com'
end

Easy for the application developer to understand. Cleaner implementation internally:

Clearance.configuration.mailer_sender

As an added bonus, in the library’s tests, we can set configuration attributes without worrying about undefining constants:

Clearance.configuration.mailer_sender= 'new.email@example.com'

Yay.

Clearance: Rails authentication for developers who write tests

Authentication is a common pattern in Rails apps. Thus, there have been many authentication plugins. We’ve tried acts_as_authenticated and restful_authentication over the years.

We found that user authentication is hard to generalize. Most abstracted authentication plugins had both too much and too little for us.

We then tried writing authentication from scratch on our clients’ Rails apps for about a year. We felt better about test coverage but it was still a pain to re-write similar code. App after app, we talked about extracting common code into a library. Each time we resisted.

After a while, we made the decision that maybe 60% of authentication could be re-used. We extracted Hoptoad’s authentication at last year’s Lone Star Ruby Conference, then merged code from two of our clients’ apps. We named the gem Clearance.

On the first attempt, we went overboard on re-use. We backed off and Mike wrote hooks in places we were finding logical extension points. For the past few months, patches have trickled in from Github and we’ve carefully included code that fits in the “60%”.

We recently started a new project. In the process, we’ve polished the gem and are happy to announce its official release.

Clearance

1. Sign up
2. Confirm email
3. Sign in
4. Sign out
5. Reset password

Get version 0.4.5 on Github.

Modules, Shoulda, & Factory Girl

Clearance is focused on maintainability of your application’s authentication code.

• Include comprehensive Shoulda & Factory Girl tests in your Rails app’s test suite.
• Encapsulate authentication logic in modules which are included in your controllers, models, & tests.

This approach keeps your Rails application’s code clean and alerts you if you ever break your authentication code.

Due to the work we’ve been doing to make Shoulda test framework-agnostic, you will be able to use RSpec in the 0.5.0 release of Clearance.

Test::Unit and Cucumber features are already supported:

script/generate clearance
script/generate clearance_features

Conventions

To keep our approach simple, we made a series of design decisions:

• User model required.
• User model uses attr_accessible.
• Authenticate by email (not username) & password.
• Users must be confirmed via email.
• Vocabulary restricted to Trinity of Signs: “sign up”, “sign in”, “sign out”

Beyond

Clearance does not try to be a Swiss Army knife but it does have some hooks if you want admin roles, sign up and sign in by username in addition to email, or something else.

Please report bugs and request features on Clearance’s Lighthouse account.

There is also a mailing list for questions.

shoulda 2.0

Lots of shoulda news to report. The 2.0.x gem has been released, which includes bug fixes, new features, and deprecations.

Rails 2.1+ install

The gem is being hosted at github.com/thoughtbot/shoulda

Rails::Initializer.run do |config|
  config.gem 'thoughtbot-shoulda', :lib => 'shoulda/rails', :source => "http://gems.github.com"
end

rake gems:install
rake gems:unpack

Now 100% gem

Listen… if you’re still a plugin, I look at you (I’m looking at you, Paperclip), shake my head, and say, “So much untapped potential. Could’ve been big.”

Why gems?

• Ruby comes with a packaging system. No need for another, especially with the gem support in Rails 2.1.
• It’s nice to scan the vendor/gems listing and see version numbers.
• github has excellent gem support.
• In shoulda’s case, it’s a project that can be used in and out of Rails. It’s a pain to have both a plugin and a gem.

Ryan McGeary and Josh Nichols did the bulk of this conversion and related refactoring. They deserve extra acclaim because they’ve made it easier for everyone to contribute to shoulda.

Improvements to current macros

should_assign_to :user, :equals => "@user"

should_assign_to :user, :class => User

should_have_one :address, :dependent => :destroy

ActiveRecord::Errors.default_error_messages is used instead of home-brewed regexes for all Active Record macros.

New Macros

context "Creating a post"
  setup { Post.create }
  should_change "Post.count", :by => 1
end

# :from and :to examples
should_change "Post.count", :from => 0, :to => 1
should_change "@post.title", :from => "old", :to => "new"

context "Updating a post"
  setup { @post.update_attributes(:title => "new") }
  should_not_change "Post.count"
end

should_filter_params :password, :ssn

should_render_with_layout # defaults to application 
should_render_with_layout 'special'
should_render_without_layout

should_route :get, "/posts", :controller => :posts, :action => :index
should_route :delete, "/posts/1", :action => :destroy, :id => 1
should_route :get, "/users/1/posts/1", 
  :action => :show, :id => 1, :user_id => 1

should_respond_with_content_type 'application/rss+xml'
should_respond_with_content_type :rss
should_respond_with_content_type /rss/

should_return_from_session :user_id, "@user.id"

Before statements

class UserControllerTest < Test::Unit::TestCase
  context "the index action" do
    setup do
      @users = [Factory(:user)]
      User.stubs(:find).returns(@users)
    end

    context "on GET" do
      setup { get :index }

      should_respond_with :success

      # runs before "get :index"
      before_should "find all users" do
        User.expects(:find).with(:all).returns(@users)
      end
    end
  end
end

Automatically load custom macros

Writing custom macros is a common practice for The Modern Shoulda Developer. For example, should_have_attached_file used to be a handy custom macro to have around when you’re using Paperclip.

There were a few problems with that, however. Where should that macro go? RAILS_ROOT/test/test_helper.rb? Why do we have the same macro in multiple apps for the same plugin?

Enter the new shoulda_macros directory. Shoulda will automatically load custom macros she finds in:

• RAILS_ROOT/test/shoulda_macros
• RAILS_ROOT/vendor/gems/#{gem_name}/shoulda_macros
• RAILS_ROOT/vendor/plugins/#{plugin_name}/shoulda_macros

Now, every Paperclip user can use the latest should_have_attached_file because it is where it belongs.

should_be_restful is being deprecated

should_be_restful will be removed in a future release. It is currently in a grace period where you’ll see a warning during your test runs. It will be added to the new woulda gem if you care to keep using it. Its problems:

• It dissuades the programmer from good TDD practice. It’s working at too high level of abstraction and encourages the programmer to take too many big steps.
• It is an attempt to be like Rails scaffolding, which is a good way to learn REST, but the syntax is so cryptic it doesn’t accomplish that goal.
• It’s near impossible to find the line in your test file that is actually failing or erroring.
• You have no idea how many tests it generates or how they’re implemented, which lulls the programmer into a false sense of security.

Removed

• load_all_fixtures (use fixtures :all instead)
• shoulda.conf and color support (use the redgreen gem instead)

Credit

Ryan McGeary, Matt Jankowski, Mike Boone, Tammer Saleh, Josh Nichols, Joe Ferris, Keith Morrison, Me