Implement flood control with redis

Question

I'm trying to replace the sql implementation of Drupal 8's flood control service with a redis based implementation.

See https://github.com/drupal/drupal/blob/8.0.x/core/lib/Drupal/Core/Flood/DatabaseBackend.php

The requirements are like this:

• Each occurrence of an action/event (e.g trying to log in) is logged with an expiration, identifier and timestamp
• I need to be able to prevent that a certain action can be done more than N times in a given timeframe
• I want to be able to clean up expired events
• In case of a threshold of 3 in 10 minutes, if the user tries once, then twice after 5 minutes, he is blocked and can try again once after 5 more minutes. Not 10. While the second would be a valid way to do this, it's not how the sql implementation works or how the tests expect it to work.
• As you can see based on the API, I also don't know when registering the event what the threshold is, I only know the expiration of a single event.

My thoughts on how to implement this:

• If, after N occurrences should be locked for the given time, then this would be easy with a single KEY for event:identifier that is incremented, once the max is reached, it is locked until it expires again and each INCR would also update the expiration (or not).
• I found many posts that ask about expiration of list entries, which is not possible. There are workarounds using sorted sets and delete by range. Most seem to use a single global set, but then I can't easily count my event + identifier - I think.

After writing all this down, I might actually have an idea how it could work, so I guess what I'm looking for is feedback on whether that makes sense or if there's an easier way.

Each event:identifier combination is a key and contains a sorted set. That uses the expiration as score and as value a unique value, possibly creation time in microseconds. I count the non-expired records to detect if the threshold was reached. I'm updating the expiration of each event:identifier to the provided expiration window, so it will be auto-deleted assuming unless a given identifier/client doesn't give up and keeps on trying, without ever reaching the expiration. Is it worth to clean up the records inside a set e.g. when doing a new register? It seems to be fairly fast, and I could also only do it sometimes.

Answer 1

I would prefer to use Redis' key expiration feature, instead of reimplementing one.

A simpler alternative would be the following one:

• just SET a simple value, which is the the number of attempts; use a key built on a pattern like "identifier":"event type" : SETNX <identifier>:<event type> 1

• if the response is 1, this is the first attempt, so you set a timeout on this key: EXPIRE <identifier>:<event type> <timeout in seconds>

• otherwise you increment the number of attempts INCR <identifier>:<event type> The response of the INCR will give you the number of attempts during the window, so you know if you can allow the action or not.

You could also use a hash instead of a simple value, if you need to store more data, like the max number of allowed attempts in the given time window. In this case you will probably use HSETNX and HINCR.