Should I trust Redis for data integrity?

Question

In my current project, I have PostgreSQL as my master DB, and Redis as kind of a slave, e.g., when some user adds another as a friend, first the relationship will be stored in PostgreSQL and then a friend list in Redis will be updated. When some user's friend list is requested, it will be pulled out of Redis instead of PostgreSQL.

The question is: when I update the friend list in Redis, should I get a fresh copy outof PostgreSQL, and replace the old list in Redis with the new one or should I keep the old list and simply SADD the userid into the list? The latter is of course best for performance, but intuitively the former does a better job in keep the data integrity? And if something like Celery is used, is the second method worth the risk?

Answer 1

This has nothing to do with Redis. When you are writing to two databases, a lot of things can go wrong even if both of them individually guarantee data integrity.

For the sake of discussion, replace Redis with MySQL in your question, and ask yourself - will data integrity be compromised?

You may have written to Postgres and then your process can die without writing to MySQL. Or perhaps there is a network outage. Or perhaps MySQL is down. In all these cases, Postgres and MySQL would start to differ.

It does not matter whether you replace the entire record or simply add one row. Both can lead to data corruption.

If you are concerned with data integrity, keep data in a single authoritative system. Otherwise, you would need a two phase commit protocol