Why is data getting stored with weird keys in Redis when using Jedis with Spring Data?

Question

I am using Spring Data Redis with Jedis. I am trying to store a hash with key vc:${list_id}. I was able to successfully insert to redis. However, when I inspect the keys using the redis-cli, I don't see the key vc:501381. Instead I see \xac\xed\x00\x05t\x00\tvc:501381.

Why is this happening and how do I change this?

Answer 1

Ok, googled around for a while and found help at http://java.dzone.com/articles/spring-data-redis.

It happened because of Java serialization.

The key serializer for redisTemplate needs to be configured to StringRedisSerializer i.e. like this:

<bean      id="jedisConnectionFactory"      class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory"      p:host-name="${redis.server}"      p:port="${redis.port}"      p:use-pool="true"/>  <bean      id="stringRedisSerializer"      class="org.springframework.data.redis.serializer.StringRedisSerializer"/>  <bean      id="redisTemplate"      class="org.springframework.data.redis.core.RedisTemplate"     p:connection-factory-ref="jedisConnectionFactory"      p:keySerializer-ref="stringRedisSerializer"     p:hashKeySerializer-ref="stringRedisSerializer"  /> 

Now the key in redis is vc:501381.

Or like @niconic says, we can also set the default serializer itself to the string serializer as follows:

<bean      id="redisTemplate"      class="org.springframework.data.redis.core.RedisTemplate"     p:connection-factory-ref="jedisConnectionFactory"      p:defaultSerializer-ref="stringRedisSerializer" /> 

which means all our keys and values are strings. Notice however that this may not be preferable, since you may want your values to be not just strings.

If your value is a domain object, then you can use Jackson serializer and configure a serializer as mentioned here i.e. like this:

<bean id="userJsonRedisSerializer" class="org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer">     <constructor-arg type="java.lang.Class" value="com.mycompany.redis.domain.User"/> </bean> 

and configure your template as:

<bean      id="redisTemplate"      class="org.springframework.data.redis.core.RedisTemplate"     p:connection-factory-ref="jedisConnectionFactory"      p:keySerializer-ref="stringRedisSerializer"     p:hashKeySerializer-ref="stringRedisSerializer"      p:valueSerialier-ref="userJsonRedisSerializer" /> 

Answer 2

I know this question has been a while, but I did some research on this topic again recently, so I would like to share how this "semi-hashed" key is generated by going thru part of the spring source code here.

First of all, Spring leverages AOP to resolve annotations like @Cacheable, @CacheEvict or @CachePut etc. The advice class is CacheInterceptor from Spring-context dependency, which is a subclass of CacheAspectSupport (also from Spring-context). For the ease of this explanation, I would use @Cacheable as an example to go thru part of the source code here.

When the method annotated as @Cacheable is invoked, AOP would route it to this method protected Collection<? extends Cache> getCaches(CacheOperationInvocationContext<CacheOperation> context, CacheResolver cacheResolver) from CacheAspectSupport class, in which it would try to resolve this @Cacheable annotation. In turn, it leads to the invocation of this method public Cache getCache(String name) in the implementing CacheManager. For this explanation, the implementing CacheManage would be RedisCacheManager (from Spring-data-redis dependency).

If the cache was not hit, it will go ahead to create the cache. Below is the key methods from RedisCacheManager:

protected Cache getMissingCache(String name) {     return this.dynamic ? createCache(name) : null; }  @SuppressWarnings("unchecked") protected RedisCache createCache(String cacheName) {     long expiration = computeExpiration(cacheName);     return new RedisCache(cacheName, (usePrefix ? cachePrefix.prefix(cacheName) : null), redisOperations, expiration,             cacheNullValues); } 

Essentially, it will instantiate an RedisCache object. To do this, it requires 4 parameters, namely, cacheName, prefix (this is the key parameter with regards to answering this question), redisOperation (aka, the configured redisTemplate), expiration (default to 0) and cacheNullValues (default to false). The constructor below shows more details about RedisCache.

/**  * Constructs a new {@link RedisCache} instance.  *  * @param name cache name  * @param prefix must not be {@literal null} or empty.  * @param redisOperations  * @param expiration  * @param allowNullValues  * @since 1.8  */ public RedisCache(String name, byte[] prefix, RedisOperations<? extends Object, ? extends Object> redisOperations,         long expiration, boolean allowNullValues) {      super(allowNullValues);      Assert.hasText(name, "CacheName must not be null or empty!");      RedisSerializer<?> serializer = redisOperations.getValueSerializer() != null ? redisOperations.getValueSerializer()             : (RedisSerializer<?>) new JdkSerializationRedisSerializer();      this.cacheMetadata = new RedisCacheMetadata(name, prefix);     this.cacheMetadata.setDefaultExpiration(expiration);     this.redisOperations = redisOperations;     this.cacheValueAccessor = new CacheValueAccessor(serializer);      if (allowNullValues) {          if (redisOperations.getValueSerializer() instanceof StringRedisSerializer                 || redisOperations.getValueSerializer() instanceof GenericToStringSerializer                 || redisOperations.getValueSerializer() instanceof JacksonJsonRedisSerializer                 || redisOperations.getValueSerializer() instanceof Jackson2JsonRedisSerializer) {             throw new IllegalArgumentException(String.format(                     "Redis does not allow keys with null value ¯\\_(ツ)_/¯. "                             + "The chosen %s does not support generic type handling and therefore cannot be used with allowNullValues enabled. "                             + "Please use a different RedisSerializer or disable null value support.",                     ClassUtils.getShortName(redisOperations.getValueSerializer().getClass())));         }     } } 

So what the use of prefix in this RedisCache? --> As shown in the constructor about, it is used in this statement this.cacheMetadata = new RedisCacheMetadata(name, prefix);, and the constructor of RedisCacheMetadata below shows more details:

/**      * @param cacheName must not be {@literal null} or empty.      * @param keyPrefix can be {@literal null}.      */     public RedisCacheMetadata(String cacheName, byte[] keyPrefix) {          Assert.hasText(cacheName, "CacheName must not be null or empty!");         this.cacheName = cacheName;         this.keyPrefix = keyPrefix;          StringRedisSerializer stringSerializer = new StringRedisSerializer();          // name of the set holding the keys         this.setOfKnownKeys = usesKeyPrefix() ? new byte[] {} : stringSerializer.serialize(cacheName + "~keys");         this.cacheLockName = stringSerializer.serialize(cacheName + "~lock");     } 

At this point, we know that some prefix parameter has been set to RedisCacheMetadata, but how exactly is this prefix used to form the key in Redis (e.g.,\xac\xed\x00\x05t\x00\tvc:501381 as you mentioned)?

Basically, the CacheInterceptor will subsequently move forward to invoke a method private RedisCacheKey getRedisCacheKey(Object key) from the above-mentioned RedisCache object, which returns an instance of RedisCacheKey by utilizing the prefix from RedisCacheMetadata and keySerializer from RedisOperation.

private RedisCacheKey getRedisCacheKey(Object key) {     return new RedisCacheKey(key).usePrefix(this.cacheMetadata.getKeyPrefix())             .withKeySerializer(redisOperations.getKeySerializer()); } 

By reaching this point, the "pre" advice of CacheInterceptor is completed, and it would go ahead to execute the actual method annotated by @Cacheable. And after completing the execution of the actual method, it will do the "post" advice of CacheInterceptor, which essentially put the result to RedisCache. Below is the method of putting the result to redis cache:

public void put(final Object key, final Object value) {      put(new RedisCacheElement(getRedisCacheKey(key), toStoreValue(value))             .expireAfter(cacheMetadata.getDefaultExpiration())); }  /**  * Add the element by adding {@link RedisCacheElement#get()} at {@link RedisCacheElement#getKeyBytes()}. If the cache  * previously contained a mapping for this {@link RedisCacheElement#getKeyBytes()}, the old value is replaced by  * {@link RedisCacheElement#get()}.  *  * @param element must not be {@literal null}.  * @since 1.5  */ public void put(RedisCacheElement element) {      Assert.notNull(element, "Element must not be null!");      redisOperations             .execute(new RedisCachePutCallback(new BinaryRedisCacheElement(element, cacheValueAccessor), cacheMetadata)); } 

Within the RedisCachePutCallback object, its callback method doInRedis() actually invoke a method to form the actual key in redis, and the method name is getKeyBytes() from RedisCacheKey instance. Below shows the details of this method:

/**  * Get the {@link Byte} representation of the given key element using prefix if available.  */ public byte[] getKeyBytes() {      byte[] rawKey = serializeKeyElement();     if (!hasPrefix()) {         return rawKey;     }      byte[] prefixedKey = Arrays.copyOf(prefix, prefix.length + rawKey.length);     System.arraycopy(rawKey, 0, prefixedKey, prefix.length, rawKey.length);      return prefixedKey; } 

As we can see in the getKeyBytes method, it utilizes both the raw key (vc:501381 in your case) and prefix key (\xac\xed\x00\x05t\x00\t in your case).