This month’s Linux Journal has an article about Redis.  I read about it while sitting on the shitter, because that’s about all that Linux Journal is useful for.  The article itself was crap, but the introduction to this product was at least tempting.

The basics:  take memcached and add a disk backing, replication, virtual memory, and some cool additional data structures.  Hashes, lists, sets, sorting, joining, transactions, yay!  I instantly got a geek boner scanning the feature list.  My boner quickly faded to about half mast when I saw the C clients that are available.  No consistent hashing?  Wait, no server hashing at all? Yet they have a gay Ruby client, fully featured?  C is the lowest common denominator when it comes to language support.  Start there, then add high level bindings using this low level library.  libmemcached got it right.  You start on the bottom and work your way up to higher level bindings.  It seems that Redis took the wrong approach to this which will result in every language having a different client implementation.  This leads to inconsistencies between languages, which is bad news.  That on first glance gave me that headed-for-the-toilet feeling for this project.

With a half-mast boner, I decided to do some benchmark comparisons.  Perhaps I’ll see some real numbers that might arouse me.  Stand back, I’m going to do some science!

The Test Setup

I used the same machine as client and server for both tests to avoid network adding a skew to the results.  (This should stress the algorithms themselves)  The hardware itself doesn’t matter, it’s an apples to apples comparison.  The software does however:

  • Redis 2.0.0 rc4
  • Memcached 1.4.5 (release)
  • Credis (libcredis client) 0.2.2
  • Libmemcached 0.31  (I know this is a little outdated, but for this test it works fine)
  • Memcache client benchmark app: mc_stress.c
  • Redis client benchmark app: redis_stress.c

The clients I wrote are just simple iterations of setting and getting keys and timing the results.  Since WP sucks for posting source code snippets I had to stuff these into PDF files.  Feel free to recreate my tests and post your results, since that’s what science is all about.

Results

This test varies the key size with a static value of only 3 bytes.  (Using 100k unique keys)  I’m guessing this will stress the internal hashing algorithms used for processing the key names.  As you can see here, Redis came up short by a range of 20-26% the speed of memcache.

Same test, but showing the Multi-GET performance.  Redis clearly has some kind of problem here, and I believe it might be due to requesting a large amount of keys in a single MGET operation.  100,000 keys seemed to be the upper limit… but in every case, Redis came up short.

This test stresses the different value sizes.  Using a small 10 byte key length, I used varying sizes of value lengths up to around 16KB.  Even though memcached advertises a 1MB value limitation, performance dropssignificantly over 8192 bytes.  SET speed was reduced by 100 times and depending on the test, GET speed was either significantly reduced, or reduced by 100 times as well.  I suspect that playing around with slab sizes might assist with the speed here, but that becomes quite the pain in the ass.  The main point:  If you use memcache, do not stuff large values into a single key.  Hash them out into namespaces, you’ll be much better off.

Adding to this, Redis boasts a 1GB value size.  We would need to graph another relationship of transfer rate to determine if there is an inefficiency in handling larger values, or if it’s just due to the size of the value itself.  Memcache shows a similar trend as value size increases, but again Redis is 20-40% slower than memcache.

Here is the same test, showing Multi-GET performance.  Similar trend as key length test.

Conclusion

After crunching all of these numbers and screwing around with the annoying intricacies of OpenOffice, I’m giving Redis a big thumbs down.  My initial sexual arousal from the feature list is long gone.  Granted, Redis might have its place in a large architecture, but certainly not a replacement to memcache.  When your site is hammering 20,000 keys per second and memcache latency is heavily dependent on delivery times, it makes no business sense to transparently drop in Redis.   The features are neat, and the extra data structures could be used to offload more RDBMS activity… but 20% is just too much to gamble on the heart of your architecture.

Maybe sometime in the future Redis will be up to par with Memcached performance.  Or, it could be the extra VM and disk backing is inherently flawed to begin with.  In that case, Memcached will always win, and Redis will be only useful in a niche market somewhere between DB caching and NoSQL fanboys.

Related Posts

« »