Redis: Data Cheeseburgers
Since we're on a NoSQL craze in Boston this week, it's about time we started talking about Redis. We'll be using it on Hoptoad extensively soon, and it's a truly useful project.
Explaining Redis is tough, it's easy to say "a data structures server" or "memcached on steroids" or something more jargon filled. It's not exactly a key value store, it's definitely not a relational or document-oriented database. The biggest selling point of Redis is that usually as programmers we have to bend our data into a table or document to save it, but with Redis we can persist data as we conceptually visualize it. Tasty!
You may have heard that Redis is super fast, is being used for everything from analytics to job queues, and even on large sites such as Craigslist. Let's take a deeper look into what Redis can do for us.
We're Rubyists, so let's use the Ruby driver courtesy of Ezra:
git clone git://github.com/ezmobius/redis-rb.git cd redis-rb rake redis:install # Clones, compiles, and installs Redis rake dtach:install # Downloads, compiles and installs Dtach rake install # Builds and installs the Redis RubyGem rake redis:start # Fires up redis-server in dtach so you can take back control of your terminal
Boom, now we should be able to connect to Redis and start messing around with it. Redis writes to disk asynchronously, so you don't need to worry about potentially losing all of your data like other in-memory stores. There's also a VM system in place to swap out values that aren't being accessed which is explained more in depth here. Check out
/etc/redis.conf for the async writing settings along with other configuration values you can tweak.
Each of the following examples do the following first to connect:
$ irb -rubygems -rredis irb(main):001:0> r = Redis.new => #>Redis:0xb7b4bb3c @password=nil, @db=0, @thread_safe=nil, @host="127.0.0.1", @sock=nil, @timeout=5, @port=6379, @logger=nil>
The "Redis" object is what redis-rb provides for us to talk to Redis. You could telnet directly to the redis-server, but let's not get too crazy here.
Redis can be considered a key/value store in this respect. Conceptually you could think of the Redis object as a giant Ruby Hash, but there's so much more to Redis than that. (Also, word on the street is there will be native support for Hash datatypes soon).
irb(main):002:0> r["fries"] = "are done" => "are done" irb(main):003:0> r["fries"] => "are done"
Just like memcached you can set a EXPIRE time on a given key, and use TTL to find out how much longer it has left to live. The annoying part is that unlike memcached, if a given key's value is changed, the expire time is reset. Also, basing functionality around EXPIRE times is sometimes necessary, but testing it is hard. We'll cover that in a future post.
irb(main):04:0> r.expire("fries", 60) => true irb(main):05:0> r.ttl("fries") => 59 irb(main):06:0> sleep(60) => 60 irb(main):07:0> r["fries"] => nil
Redis also offers atomic increments and decrements of numbers, which makes it a perfect fit for download/hit/visitor counters. Atomic modifications means that you don't have to worry about locking or conflict issues with multiple clients connecting to Redis and performing the same operation on the same keys.
irb(main):08:0> r.incr("orders") => 1 irb(main):09:0> r.incr("orders", 5) => 6 irb(main):10:0> r.decr("orders") => 5
This is the first "data structure" like operation that Redis supports, and what really sets it apart from any other NoSQL project that I know of. With Redis you can set up simple lists and use them like a queue, a stack, or however you want. With these commands, the first argument is the name of the key, and the second is the value we're giving to the list.
irb(main):011:0> r.push_tail "fiveguys", "milkshake" => "OK" irb(main):012:0> r.push_tail "fiveguys", "fries" => "OK" irb(main):013:0> r.push_tail "fiveguys", "burgers" => "OK" irb(main):014:0> r.list_range "fiveguys", 0, -1 => ["milkshake", "fries", "burgers"] irb(main):015:0> r.pop_tail "fiveguys" => "burgers" irb(main):016:0> r.pop_head "fiveguys" => "milkshake"
There's a lot of cool features with lists that I'll explain in future articles, but this is really just the tip of the iceberg.
You guessed it: lists with guaranteed unique elements. The nice things about sets is that you can natively check if a given element is in a set, and also do a union, intersection, or diff between sets if need be.
irb(main):017:0> r.set_add "nick_orders", "hamburger" => true irb(main):018:0> r.set_add "nick_orders", "hamburger" => false irb(main):019:0> r.set_add "nick_orders", "hotdog" => true irb(main):020:0> r.set_add "nick_orders", "cheeseburger" => true irb(main):021:0> r.set_members "nick_orders" => ["cheeseburger", "hotdog", "hamburger"] irb(main):022:0> r.set_member? "nick_orders", "hamburger" => true irb(main):023:0> r.set_add "ralph_orders", "hamburger" => true irb(main):024:0> r.set_add "ralph_orders", "fries" => true irb(main):025:0> r.set_intersect "nick_orders", "ralph_orders" => ["hamburger"]
Finally, there's sorted sets. I really didn't get this at first because of the name. This data type stores unique elements that each have an associated score. You can atomically increment the score, remove multiple elements based on score, and so on.
irb(main):026:0> r.zset_add "menu", 4.99, "cheeseburger" => true irb(main):027:0> r.zscore "menu", "cheeseburger" => "4.9900000000000002" # not sure where this rounding error came from! irb(main):028:0> r.zset_add "menu", 2.99, "fries" => true irb(main):029:0> r.zset_add "menu", 1.99, "coke" => true irb(main):030:0> r.zrange "menu", 0, -1 => ["coke", "fries", "cheeseburger"] irb(main):031:0> r.zincrby "menu", 5, "fries" => "7.9900000000000002" irb(main):032:0> r.zrange "menu", 0, -1 => ["coke", "cheeseburger", "fries"] irb(main):033:0> r.zrangebyscore "menu", 2, 10 => ["cheeseburger", "fries"]
Here's some other material to get you started:
- Data Types Intro
- redis-rb examples
- Try Redis in your browser
- A collection of Redis Use Cases
- My Sinatra/Redis boggle clone
If you know of other great starting resources for those new to Redis, feel free to let us know in the comments.
Next Steps & Related Reading
Redis sets - The intersection of space and time
Redis Pub/Sub…how does it work?