Redis: Data Cheeseburgers

Since we’re on a NoSQL craze in Boston this week, it’s about time we started talking about Redis. We’ll be using it on Hoptoad extensively soon, and it’s a truly useful project.

Explaining Redis is tough, it’s easy to say “a data structures server” or “memcached on steroids” or something more jargon filled. It’s not exactly a key value store, it’s definitely not a relational or document-oriented database. The biggest selling point of Redis is that usually as programmers we have to bend our data into a table or document to save it, but with Redis we can persist data as we conceptually visualize it. Tasty!

You may have heard that Redis is super fast, is being used for everything from analytics to job queues, and even on large sites such as Craigslist. Let’s take a deeper look into what Redis can do for us.

Install

We’re Rubyists, so let’s use the Ruby driver courtesy of Ezra:

git clone git://github.com/ezmobius/redis-rb.git
cd redis-rb
rake redis:install # Clones, compiles, and installs Redis
rake dtach:install # Downloads, compiles and installs Dtach
rake install       # Builds and installs the Redis RubyGem
rake redis:start   # Fires up redis-server in dtach so you can take back control of your terminal

Boom, now we should be able to connect to Redis and start messing around with it. Redis writes to disk asynchronously, so you don’t need to worry about potentially losing all of your data like other in-memory stores. There’s also a VM system in place to swap out values that aren’t being accessed which is explained more in depth here. Check out /etc/redis.conf for the async writing settings along with other configuration values you can tweak. Each of the following examples do the following first to connect:

$ irb -rubygems -rredis
irb(main):001:0> r = Redis.new
=> #>Redis:0xb7b4bb3c @password=nil, @db=0, @thread_safe=nil, @host="127.0.0.1", @sock=nil, @timeout=5, @port=6379, @logger=nil>

The “Redis” object is what redis-rb provides for us to talk to Redis. You could telnet directly to the redis-server, but let’s not get too crazy here.

Keys

Redis can be considered a key/value store in this respect. Conceptually you could think of the Redis object as a giant Ruby Hash, but there’s so much more to Redis than that. (Also, word on the street is there will be native support for Hash datatypes soon).

irb(main):002:0> r["fries"] = "are done"
=> "are done"

irb(main):003:0> r["fries"]
=> "are done"

Expiration

Just like memcached you can set a EXPIRE time on a given key, and use TTL to find out how much longer it has left to live. The annoying part is that unlike memcached, if a given key’s value is changed, the expire time is reset. Also, basing functionality around EXPIRE times is sometimes necessary, but testing it is hard. We’ll cover that in a future post.

irb(main):04:0> r.expire("fries", 60)
=> true

irb(main):05:0> r.ttl("fries")
=> 59

irb(main):06:0> sleep(60)
=> 60

irb(main):07:0> r["fries"]
=> nil

Counters

Redis also offers atomic increments and decrements of numbers, which makes it a perfect fit for download/hit/visitor counters. Atomic modifications means that you don’t have to worry about locking or conflict issues with multiple clients connecting to Redis and performing the same operation on the same keys.

irb(main):08:0> r.incr("orders")
=> 1

irb(main):09:0> r.incr("orders", 5)
=> 6

irb(main):10:0> r.decr("orders")  
=> 5

Lists

This is the first “data structure” like operation that Redis supports, and what really sets it apart from any other NoSQL project that I know of. With Redis you can set up simple lists and use them like a queue, a stack, or however you want. With these commands, the first argument is the name of the key, and the second is the value we’re giving to the list.

irb(main):011:0> r.push_tail "fiveguys", "milkshake"
=> "OK"

irb(main):012:0> r.push_tail "fiveguys", "fries"    
=> "OK"

irb(main):013:0> r.push_tail "fiveguys", "burgers"
=> "OK"

irb(main):014:0> r.list_range "fiveguys", 0, -1
=> ["milkshake", "fries", "burgers"]

irb(main):015:0> r.pop_tail "fiveguys"
=> "burgers"

irb(main):016:0> r.pop_head "fiveguys"
=> "milkshake"

There’s a lot of cool features with lists that I’ll explain in future articles, but this is really just the tip of the iceberg.

Sets

You guessed it: lists with guaranteed unique elements. The nice things about sets is that you can natively check if a given element is in a set, and also do a union, intersection, or diff between sets if need be.

irb(main):017:0> r.set_add "nick_orders", "hamburger"
=> true

irb(main):018:0> r.set_add "nick_orders", "hamburger"
=> false

irb(main):019:0> r.set_add "nick_orders", "hotdog"   
=> true

irb(main):020:0> r.set_add "nick_orders", "cheeseburger"
=> true

irb(main):021:0> r.set_members "nick_orders"
=> ["cheeseburger", "hotdog", "hamburger"]

irb(main):022:0> r.set_member? "nick_orders", "hamburger"
=> true

irb(main):023:0> r.set_add "ralph_orders", "hamburger"
=> true

irb(main):024:0> r.set_add "ralph_orders", "fries"
=> true

irb(main):025:0> r.set_intersect "nick_orders", "ralph_orders"
=> ["hamburger"]

Sorted Sets

Finally, there’s sorted sets. I really didn’t get this at first because of the name. This data type stores unique elements that each have an associated score. You can atomically increment the score, remove multiple elements based on score, and so on.

irb(main):026:0> r.zset_add "menu", 4.99, "cheeseburger"
=> true

irb(main):027:0> r.zscore "menu", "cheeseburger"
=> "4.9900000000000002" # not sure where this rounding error came from!

irb(main):028:0> r.zset_add "menu", 2.99, "fries"
=> true

irb(main):029:0> r.zset_add "menu", 1.99, "coke"
=> true

irb(main):030:0> r.zrange "menu", 0, -1
=> ["coke", "fries", "cheeseburger"]

irb(main):031:0> r.zincrby "menu", 5, "fries"
=> "7.9900000000000002"

irb(main):032:0> r.zrange "menu", 0, -1
=> ["coke", "cheeseburger", "fries"]

irb(main):033:0> r.zrangebyscore "menu", 2, 10
=> ["cheeseburger", "fries"]

Get started!

Here’s some other material to get you started:

If you know of other great starting resources for those new to Redis, feel free to let us know in the comments.

thoughtbot

giant robots smashing into other giant robots

We are thoughtbot, a web design and development agency in Boston, MA.