How to bulk delete hundreds of thousands of keys with special characters in Redis

Question

We have a list of hundreds of thousands of Redis keys containing all sorts of special characters and we would like to bulk delete them. There are some great answers to a similar problem on this question: How to atomically delete keys matching a pattern using Redis

HOWEVER, I can't seem to find an answer for the case where:

1. We have a large number of keys (hundreds of thousands)
2. The keys have all manners of special characters like double quotes ("), backslashes (), all sorts of weird Unicode characters, etc.
3. We are using the windows redis-cli client
4. Bonus: Ideally we would be able to issue this command as part of a MULTI/EXEC transaction so we can also delete a SET atomically along with the keys.

I would LOVE if we could just do something like the below, but have it handle keys with all of the special characters that give Redis problems:

redis-cli SMEMBERS "myGiganticListOfKeys" | xargs --delim='\n' redis-cli DEL

Unfortunately this just gives the below error:

"C:/Program Files (x86)/Git/bin/xargs.exe": redis-cli: Bad file number

I think this would otherwise work if we didn't have special characters in the keys.

Thanks so much in advance.

Answer 1

You should try creating an application using a more robust client. See the client list. The redis-cli is a very basic command-line utility intended just to hack/play with Redis.

I agree with you that the best will be to re-design your key/value store.

Consider using a tagging mechanism if you need to invalidate multiple keys: Use a hash to group keys by tag when the keys are added, and then invalidate the entire tag removing all the keys in the hash.