Spring Redis - Indexes not deleted after main entry expires

Question

I am saving new entries with a Spring Data Repository. I have a TTL of 10 seconds for each entry.

When I save an entry with indexes, here is what i get in Redis

127.0.0.1:6379> keys *
1) "job:campaignId:aa"
2) "job:a6d6e491-5d75-4fd0-bd8e-71692f6d18be"
3) "job:recipient:dd"
4) "job:a6d6e491-5d75-4fd0-bd8e-71692f6d18be:phantom"
5) "job:listId:cc"
6) "job:accountId:bb"
7) "job"
8) "job:a6d6e491-5d75-4fd0-bd8e-71692f6d18be:idx"

After the expiration, I still have data :

127.0.0.1:6379> keys *
1) "job:campaignId:aa"
2) "job:recipient:dd"
3) "job:listId:cc"
4) "job:accountId:bb"
5) "job"
6) "job:a6d6e491-5d75-4fd0-bd8e-71692f6d18be:idx"

Without any TTL.

Why aren't they deleting themself ? How could I do that ?

Answer 1

Spring Data Redis Repositories use multiple Redis features to persist domain objects in Redis.

Domain objects are stored primarily in a hash (job:a6d6e491-5d75-4fd0-bd8e-71692f6d18be). Any expiry is applied directly to the hash so Redis can expire the key. Spring Data Redis also maintains secondary indexes (job:campaignId:aa, job:recipient:dd) to provide lookup by particular field values. Individual elements inside a set cannot be expired. Only the whole data structure can expire, but that's not the thing you want to do because all non-expired elements would disappear that way.

So Spring Data Redis persists a copy of the original hash as phantom hash (job:a6d6e491-5d75-4fd0-bd8e-71692f6d18be:phantom) with a slightly longer TTL.

Spring Data Redis subscribes to key-events (with setting @EnableRedisRepositories(enableKeyspaceEvents = EnableKeyspaceEvents.ON_STARTUP) to listen to expiry events. As soon as the original hash expires, Spring Data Redis loads the phantom hash to perform cleanups (remove references from secondary indexes).

The reason why the cleanup of your data wasn't performed can have multiple reasons:

1. If you run a console application just to insert data and terminate, then the expiry removes the hashes but does not perform the index cleanup since your application is not running anymore. Any events published by Redis are transient, and if your application is not listening, then these events are lost
2. If you have enabled repository support with just @EnableRedisRepositories (without enabling keyspace-events), then the Keyspace event listener is not active, and Spring Data Redis is not subscribed to any expiry events.