Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Stale data handling with memcached/consistent hashing

Assume I have two memcached nodes (node A, B) at the beginning, and when I add a new node C, a portion of the keys are remapped and thanks to consistent hashing only some of them.

Let's assume a value with key "foo" originally at server A is now being to mapped to server C.

When I finally remove node C, the key should be mapped to node A again, but at that time node A only contains stale data.

So, is flushing the data the only way to solve this issue?

like image 741
Ryan Avatar asked Dec 19 '12 07:12

Ryan


People also ask

Does memcached use consistent hashing?

One of the most important features needed to create a scalable memcached infrastructure is consistent hashing.

Is Memcache consistent?

Strong consistency between Memcached replicas guarantees that Memcached replicas have the same data and that different read requests to the replicas will return the same result.

What problem does consistent hashing solve?

Consistent hashing solves the horizontal scalability problem by ensuring that every time we scale up or down, we DO NOT have to re-arrange all the keys or touch all the database servers.

Which databases use consistent hashing?

Consistent hashing is also used for partitioning in Amazon's Dynamo storage system, by the Riak key-value database, and as part of the Akamai Content Delivery Network.


2 Answers

I think the only thing you have to do with stale data is...nothing! you know that if you introduce a new server, part of cache will be invalidated. that's all! let memcached to get rid of stale data for you when that moment comes!

another thing is that you cannot get rid of stale objects by yourself, because you don t know which server store that object(thank to consistent hashing algorithm).

hope to answered your question.

like image 115
andrea.marangoni Avatar answered Sep 17 '22 23:09

andrea.marangoni


Your problem boils down to "data in your cache is stale"; so how to correct it?

So you need to update the cache with latest value. I don't think there is any alternative to this. You can consider different approaches to optimize it depending on your system requirement.

like image 32
rai.skumar Avatar answered Sep 21 '22 23:09

rai.skumar