Fastest database engine for caching?

Question

I use MySQL for my primary database, where I keep the actual objects. When an object is rendered using a template, rendering takes a lot of time.

Because of that I've decided to cache the produced HTML. Right now I store the cache in files, named appropriate, and it does work significantly faster. I am however aware that it is not the best way to do so.

I need a (preferably key-value) database to store my cache in. I cannot use a caching proxy because I still need to process the cached HTML. Is there such a database with a PHP front end?

Edit: If I use memcached, and I cache about a million pages, won't I run out of RAM?

Edit 2: And again, I have a lot of HTML to cache (gigabytes of it).

Answer 1

If I use memcached, and I cache about a million pages, won't I run out of RAM?

Memcached

memcached is also a real solid product(like redis more) used at all big sites to keep them up and running. Almost al active tweets(which user fetch) are stored in memcached for insane performance.

If you want to be fast you should have your active dataset in memory. But yeah if the dataset is bigger then your available memory you should(should always store data in persistent datastore because memcached is volatile) store data in a persistent datastore like for example mysql. When it's not available in memory you will try and fetch it from datastore and cache it memcache for future reference(with expire header).

Redis

I really like redis because it is an advanced key-value store with insane performance

Redis is an advanced key-value store. It is similar to memcached but the dataset is not volatile, and values can be strings, exactly like in memcached, but also lists, sets, and ordered sets. All this data types can be manipulated with atomic operations to push/pop elements, add/remove elements, perform server side union, intersection, difference between sets, and so forth. Redis supports different kind of sorting abilities.

Redis has a VM so you don't need a seperate persisent datastore. I really like redis because of all the available commands (power :)?). This tutorial by simon willison displays(a lot of) the raw power which redis has.

Speed

Redis is pretty fast!, 110000 SETs/second, 81000 GETs/second in an entry level Linux box. Check the benchmarks.

Commits

Redis is more actively developed. 8 hours ago antirez(redis) commited something versus memcached 12 November latest commit.

Install Redis

Redis is insanely easy to install. It has no dependencies. You only have to perform:

make
./redis-server redis.conf #start redis

to compile redis(Awesome :)?).

Install Memcached

Memcached has dependency(libevent) which makes it more difficult to install.

wget http://memcached.org/latest
tar -zxvf memcached-1.x.x.tar.gz
cd memcached-1.x.x
./configure
make && make test
sudo make install

not totally true because memcached has libevent dependency and ./configure will fail of libevent is missing. But then again they have packages which are cool, but require root to install.

Answer 2

Redis is pretty fast: 110,000 SETs/second

If speed is a concern, why use the network layer?

According to: http://tokutek.com/downloads/mysqluc-2010-fractal-trees.pdf

• InnoDB inserts ....................43,000 records per second AT ITS PEAK*;
• TokuDB inserts ....................34,000 records per second AT ITS PEAK*;
• G-WAN KV inserts ....100,000,000 records per second

(*) after a few thousands of inserts, performances degrade severely for InnoDB and TokuDB which end to write to disk when their cache and the system cache and the disk controller cache are full. See the PDF for an interesting discussion of the problems caused by the topology of the InnoDB database index (which severely breaks locality while the Fractals topology scales much better... but still not linearly).