Is Redis atomic when multiple clients attempt to read/write an item at the same time?

Question

Let's say that I have several AWS Lambda functions that make up my API. One of the functions reads a specific value from a specific key on a single Redis node. The business logic goes as follows:

if the key exists:
    serve the value of that key to the client
if the key does not exist:
    get the most recent item from dynamoDB
    insert that item as the value for that key, and set an expiration time
    delete that item from dynamoDB, so that it only gets read into memory once
    Serve the value of that key to the client

The idea is that every time a client makes a request, they get the value they need. If the key has expired, then lambda needs to first get the item from the database and put it back into Redis.

But what happens if 2 clients make an API call to lambda simultaneously? Will both lambda processes read that there is no key, and both will take an item from a database?

My goal is to implement a queue where a certain item lives in memory for only X amount of time, and as soon as that item expires, the next item should be pulled from the database, and when it is pulled, it should also be deleted so that it won't be pulled again.

I'm trying to see if there's a way to do this without having a separate EC2 process that's just keeping track of timing.

Is redis+lambda+dynamoDB a good setup for what I'm trying to accomplish, or are there better ways?

Answer 1

A Redis server will execute commands (or transactions, or scripts) atomically. But a sequence of operations involving separate services (e.g. Redis and DynamoDB) will not be atomic.

One approach is to make them atomic by adding some kind of lock around your business logic. This can be done with Redis, for example.

However, that's a costly and rather cumbersome solution, so if possible it's better to simply design your business logic to be resilient in the face of concurrent operations. To do that you have to look at the steps and imagine what can happen if multiple clients are running at the same time.

In your case, the flaw I can see is that two values can be read and deleted from DynamoDB, one writing over the other in Redis. That can be avoided by using Redis's SETNX (SET if Not eXists) command. Something like this:

GET the key from Redis
If the value exists:
    Serve the value to the client
If the value does not exist:
    Get the most recent item from DynamoDB
    Insert that item into Redis with SETNX
        If the key already exists, go back to step 1
    Set an expiration time with EXPIRE
    Delete that item from DynamoDB
    Serve the value to the client