RedisHash - this hash is a db!

I started to understand how [Redis] is working and how can I use it in my projects. In my current project I collect some informations with background jobs, and store these informations in Redis. I choosen this way because these data should not be really persistent (that’s why I not decided to use MySQL as storage) because it can be collected quickly again if vanishes (it collected periodically) but I would like to access it quickly.

Redis has a beautiful interface to handle data. However, I usually like if the data storing logic is separated from my business logic. After some thinking I decided to store object in a easiest way I can: I store them in a “smart” hash. Basically, all hashes are key-value store, and Redis is a key-value store too. So, I extended ruby’s Hash class, and overriden some methods…

RedisHash code
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
require 'rubygems'
require 'redis'
require 'json'

class RedisHash < Hash

  attr_reader :redis, :redkey

  METHODS = [:append, :blpop, :brpop, :brpoplpush, :decr, :decrby,
             :del, :exists, :expire, :expireat, :get, :getbit, :getrange, :getset,
             :hdel, :hexists, :hget, :hgetall, :hincrby, :hkeys, :hlen, :hmget,
             :hmset, :hset, :hsetnx, :hvals, :incr, :incrby, :lindex, :linsert,
             :llen, :lpop, :lpush, :lpushx, :lrange, :lrem, :lset, :ltrim, :move,
             :persist, :ping, :publish, :rename, :renamenx, :rpop, :rpoplpush, :rpush,
             :rpushx, :sadd, :scard, :sdiff, :sdiffstore, :set, :setbit, :setex,
             :setnx, :setrange, :sinter, :sinterstore, :sismember, :smembers,
             :smove, :sort, :spop, :srandmember, :srem, :strlen, :subscribe,
             :sunion, :sunionstore, :ttl, :type, :unsubscribe, :watch, :zadd,
             :zcard, :zcount, :zincrby, :zinterstore, :zrange, :zrangebyscore,
             :zrank, :zrem, :zremrangebyrank, :zremrangebyscore, :zrevrange,
             :zrevrangebyscore, :zrevrank, :zscore, :zunionstore]


  def initialize(key, redis = Redis.current)
    key = key.to_redis_key if key.respond_to?(:to_redis_key)
    @redkey = key
    @redis = redis
  end

  # Mimics the similar hash method, but gets value from Redis
  # @param [Object] key Key in the sky
  # @return [Object] a value for key
  def [](key)
    key = key.to_redis_key if key.respond_to?(:to_redis_key)
    realkey = "#{redkey}:#{key}"
    value = @redis.get(realkey)
    value.nil? ? nil : JSON.parse(value) rescue eval(value)
  end

  # Mimics the similar hash method, but sets the value in Redis
  # @param [Object] key   Key in the sky
  # @param [Object] value a value for key
  # @return [Object] an passed value
  def []=(key, value)
    key = key.to_redis_key if key.respond_to?(:to_redis_key)
    realkey = "#{redkey}:#{key}"
    @redis.set(realkey, value.to_json)
    value
  end

  # Mimics the similar hash method, but gets the list of the keys from Redis
  # @return [Array] array of keys
  def keys
    # Because a limitation of this implementation, we filter out keys what not in our scope
    redis.keys.find_all { |k| k.starts_with?(redkey.to_s) }.collect { |k| k.gsub(/^#{redkey}:/, "") }
  end

  # Mimics the similar hash method, but removes the key from Redis
  def delete(key)
    realkey = "#{redkey}:#{key}"
    oldval = self[key]
    @redis.del(realkey)
    oldval
  end

  def clear
    each_key { |k| delete(k) }
  end

  def each_key(&block)
    keys.each do |key|
      yield(key)
    end
  end


  def inspect
    s = "RedisHash(%s)"
    keyz = keys[0..9].collect { |k| k.to_sym.inspect }.join(', ')
    keyz << ", ..." if keys.size > 10
    sprintf(s, keyz)
  end

  METHODS.each do |meth|
    class_eval <<-EVAL
      def #{meth}(*args, &block)
        redis.send(:#{meth}, self, *args, &block)
      end
    EVAL
  end
end

It is a really naive implementation and I didn’t checked all hash methods to be working against this class, but a most used ones are working correctly.

A special thing where this class is differs from “normal” hashes is this is needs an initialization via constructor.

RedisHash’s constructor takes two parameter. First is a symbol, this will identify our hash. Be careful when you choose this key: it should be unique in the Redis server because if you reconnect again to this hash, you can get stored data again. It works like a database name in MySQL.

The second parameter is an existing Redis connection. If you do not pass it, be careful you have to initialize Redis.current connection before you call anything on this hash.

Take the following example:

Collection server
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def count_log_files(dir)
  rdb = RedisHash.new(:dirs)

  if File.directory?(dir)
    entries = Dir.glob("#{dir}/*")

    entries.each do |entry|
      collect_log_files(entry)
    end
  else
    file = dir
    dir = File.dirname(file)
    if file =~ /\.log$/
      rdb[dir] = rdb.key?(dir) ? 1 : rdb[dir] + 1
    end
  end
end
Collection client
1
2
3
4
5
rdb = RedisHash.new(:dirs)

rdb.each_pair do |dir, count|
  puts "#{dir} has #{count} log(s)"
end

The two service can run separatedly (on separated server too) without knowing anything about each other. In both server and client side after initialization, you can treat RedisHash as a normal ruby hash - you can assign or query a value or iterate over keys and values. And these operations immediatelly executed on your Redis db.

If you note, I defined some methods on the hash what can help us to manage our Redis db. It can be useful if you want to get or set some special value what stored in your DB.

Comments