Home > Uncategorized > Re: Using Redis as a Cache Backend in Magento

Re: Using Redis as a Cache Backend in Magento

August 14th, 2013

The fine folks at the Magento ECG have published a nice article on using Redis as a cache backend in Magento. It provides some nice benchmarks and setup tips but there is one pretty major point where I disagree with their advice: using a single Redis instance. Here is why:

  1. Redis is single-threaded so it’s performance peaks when it uses one full CPU core. On a multi-core machine (every machine these days) your peak performance with Redis will be higher if you can use multiple cores. Also, there will be no waiting between different processes. E.g. while one process may be fetching the config, another would be waiting to load a session and another would be waiting to load a FPC record. By having each on a separate core you have much higher peak performance and less contention.
  2. Redis allows you to set maxmemory but this applies to the entire process so when using multiple databases with maxmemory of say 1GB, there is no way to prevent the session backend from using 999MB and leave only 1MB for the FPC or vice-versa. So having separate instances of Redis means you can allocate maxmemory separately as appropriate.
  3. Driver bugs.. AFAIK the Cm_Cache_Backend_Redis plays well with db numbers besides 0 but the phpredis driver is actually pretty buggy and especially old versions have problems where if the connection is lost the driver will reconnect but when it reconnects it will not select the last used database and just default to 0. So for example if your FPC is 0 and the sessions are in 1 and the connection reconnects you may end up with sessions in your FPC database. These are silly bugs to have but the safest thing to do is run a separate process for each and always use db 0.
  4. Stats. Redis stores stats globally so to see for example number of evictions or peak size or something these stats would be useless if you have multiple dbs in one instance.
  5. Configuration. You cannot configure Redis settings like eviction policy, disk flushing, etc. per-database but it may make sense to use different settings for different purposes. E.g. I probably wouldn’t bother persisting FPC records to disk at all, or perhaps very infrequently, but I would definitely persist sessions.
  6. Redis processes are extremely lightweight, around 1MB of memory when empty so there is really no reason not to configure multiple instances. Just copy the init/upstart script and tweak it as many times as needed.

I think it is also worth mentioning that the README.md file is worth a read; those poor Enterprise Edition folks didn’t get a copy of that with their shiny new 1.13. ;-)

Uncategorized

  • Alex Samorukov

    Hi Colin,

    Thank you very much for a valuable feedback (and for Redis module itself, of course), i am completely agree with your point about many REDIS instances, we got the same feeling on a last testings, but article itself was already finalized.

    I am planning to update an article with this (and not only) in the nearest weeks.

    P.S. i personally prefer to store sessions in memcached, because i don`t see too much benefits in using Redis for that.

  • http://colin.mollenhour.com Colin Mollenhour

    Regarding Redis vs memcached, I think persistence is a very important requirement for sessions to avoid lost sales if a server gets rebooted for example. Other advantages of Redis over memcached are more eviction policy options and on the fly configuration. Of course the single threaded design is the big disadvantage.

  • Sebastian Po

    Colin thanks for your work and wisdom!
    One question re: sessions, what about race conditions between different requests against the same session? We have an issue with memcached locking the session, and having other requests waiting even if these only want to read from the session (of course our dear friend Ajax is involved in this mess). Would Redis solve this scenario better than memcached?

  • http://colin.mollenhour.com Colin Mollenhour

    There is no mechanism in PHP to specify a session be read-only or read-write, but one thing you can do is close the session early using session_write_close so that the lock is freed. Also, try Cm_RedisSession, it has a feature which allows you to disable locking per-request by calling “define(‘CM_REDISSESSION_LOCKING_ENABLED’, FALSE);” before the session is initialized. This won’t be read-only but it will prevent lock waiting.

  • http://colin.mollenhour.com Colin Mollenhour

    Regarding Redis vs memcached, I think persistence is a very important requirement for sessions to avoid lost sales if a server gets rebooted for example. Other advantages of Redis over memcached are more eviction policy options and on the fly configuration. Of course the single threaded design is the big disadvantage.

  • Alex Samorukov

    Actually it is possible to get persistence using memcache API using couchbase, MySQL with memcached API or some other key->value servers with memcache protocol support. About eviction – its controversial, i think default algorithm in memcached (oldest records removed first) is great for sessions, and what is more important – its not affecting performance so hard as Redis evictions. Another advantage is written on C session handler with locking support, so potentially less CPU/memory used for this operations.

  • Anonymous

    Thanks Colin, we follow the same rules and agree with you as well. I have created a series of config and init scrips for CentOS/REHL for Redis and the multiple instances. Hopefully it will save people some time to get up and running. Feedback is welcome. https://github.com/adamlevenson/redis-magento-config

  • http://colin.mollenhour.com Colin Mollenhour

    Very nice, thanks for sharing. I’m more of a Debian guy personally but that’ll certainly save the Redhat folks some time.

  • Anonymous

    Sure. Yeah, we used to be but all our client’s production servers are on RHEL so we moved to CentOS for everything else. Keep up the good work!

  • http://magemojo.com/blog/?p=31 Magento Cache Performance / MageMojo Blog
  • http://www.interactiv4.com/blog-es/magento-ecommerce/actualizacion-magento-community-1-8/ Actualización de Magento Community 1.8 by Tom Robertshaw | interactiv4.com

    […] un momento divertido en el que se explicó cómo Colin Mollenhour’s usó Redis como cache en el backend y se renombró al incorporarlo al Core de Magento. Pero ya ha sido solucionado y Piotr se disculpó […]

  • MrSicks

    First thanks for your work Colin ! I’m currently trying to install it but there’s no key when I look in redis-cli :/ Do you have any suggestion for me ?

  • http://dluat.com/aws-elastic-load-balancer-creating-cache-misses/ AWS Elastic Load Balancer Creating Cache Misses? | DL-UAT

    […] you are more likely to have contention and waits for access to the data. See Colin’s “Re: Using Redis as a Cache Backend in Magento” for more information on this […]

  • http://www.domaratonu.blogger.com Radosław Zagórski

    Hi Colin, i have one server with magento where are two magento instances (production and test). I started to use redis in backend for both and there is a problem. Redis has a tags for both instance which first create a tag in cache is write to redis. Is it possible to change redis key for instances?

  • http://colin.mollenhour.com/ Colin M

    It *should* be safe to use the same instance for two stores due to Mage_Core_Model_Cache and the cache_id_prefix, but I still would not recommend it. At minimum I would recommend using a different database index for each, but for best performance I would recommend entirely separate Redis instances for each environment. This requires creating extra init scripts and using different port numbers or unix sockets, but is worth the effort, IMO.

  • http://www.domaratonu.blogger.com Radosław Zagórski

    Thanks, I use seperate Redis instances for env. And it works great!

    But i found sth strange when i use backend redis in redis db I see a lot of redis keys (eg. “zc:k:3ea_CONFIG_GLOBAL_CRONTAB”) when i use in this time frontend action (eg. add product to basket) the backend keys disappearing and i see only frontend keys (eg. “zc:k:3ea_DB_PDO_MYSQL_DDL_sales_flat_quote_item_1”) is it correct?

  • http://colin.mollenhour.com/ Colin M

    Sounds like something is wrong but I have no idea what…