Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to disable Redis Caching at run time if redis connection failed

We have rest api application. We use redis for API response caching and internal method caching. If redis connection then it is making our API down. We want to bypass the redis caching if that redis connection fails or any exception instead of making our API down. There is a interface CacheErrorHandler but it handles the redis get set operation failures not redis connection problems. We are using Spring 4.1.2.

like image 487
cooldude Avatar asked Mar 12 '15 07:03

cooldude


People also ask

How do I disable Redis caching?

A: You can disable the Redis Object Cache from the WordPress Settings. Go to Settings > Redis, and click on the Disable Object Cache.

How do you handle Redis failure?

If the Redis server becomes unavailable you must flush it (clean out the entries) before making it available again. Otherwise there's a chance that you might start retrieving cache entries that have incorrect data because of updates that occurred in the meantime.

What happens if Redis fails?

If the connection has failed, or is closed, for standard cache ops, the (Redis)CacheManager could return an instance of RedisCache for getCache(String name) that always returns null (indicating a Cache miss on an entry), thus passing through to the underlying data store.

What happens if Redis server goes down?

When Redis goes down, you have to deal with it. It is the same as if your file system is gone or your SQL Server is down. A lot of those systems disable all writes and set their cluster in maintenance state.


1 Answers

Let's boil this down a bit. Your application uses caching (implemented with Redis). If the Redis connection is stale/closed or otherwise, then you want the application to bypass caching and (presumably) go directly to an underlying data store (e.g. RDBMS). The application Service logic might look similar to...

@Service
class CustomerService ... {

    @Autowired
    private CustomerRepository customerRepo;

    protected CustomerRepository getCustomerRepo() {
        Assert.notNull(customerRepo, "The CustomerRepository was not initialized!");
        return customerRepo;
    }

    @Cacheable(value = "Customers")
    public Customer getCustomer(Long customerId) {
        return getCustomerRepo().load(customerId);
    }
    ...
}

All that matters in Spring core's Caching Abstraction to ascertain a Cache "miss" is that the value returned is null. As such, Spring Caching Infrastructure will then proceed in calling the actual Service method (i.e. getCustomer). Keep in mind on the return of the getCustomerRepo().load(customerId) call, you also need to handle the case where Spring's Caching Infrastructure attempts to now cache the value.

In the spirit of keeping it simple, we will do without AOP, but you should be able to achieve this using AOP as well (your choice).

All you (should) need is a "custom" RedisCacheManager extending the SDR CacheManager implementation, something like...

package example;

import org.springframework.cache.Cache;
import org.springframework.data.redis.cache.RedisCacheManager;
...

class MyCustomRedisCacheManager extends RedisCacheManager {

    public MyCustomerRedisCacheManager(RedisTemplate redisTemplate) {
        super(redisTemplate);
    }

    @Override
    public Cache getCache(String name) {
        return new RedisCacheWrapper(super.getCache(name));
    }


    protected static class RedisCacheWrapper implements Cache {

        private final Cache delegate;

        public RedisCacheWrapper(Cache redisCache) {
            Assert.notNull(redisCache, "'delegate' must not be null");
            this.delegate = redisCache;
        }

        @Override
        public Cache.ValueWrapper get(Object key) {
            try {
              delegate.get(key);
            }
            catch (Exception e) {
                return handleErrors(e);
            }
        }

        @Override
        public void put(Object key, Object value) {
            try {
                delegate.put(key, value);
            }
            catch (Exception e) {
                handleErrors(e);
            }
        }

        // implement clear(), evict(key), get(key, type), getName(), getNativeCache(), putIfAbsent(key, value) accordingly (delegating to the delegate).

        protected <T> T handleErrors(Exception e) throws Exception {
            if (e instanceof <some RedisConnection Exception type>) {
                // log the connection problem
                return null;
            }
            else if (<something different>) { // act appropriately }
            ...
            else {
                throw e;
            }
        }
    }
}

So, if Redis is unavailable, perhaps the best you can do is log the problem and proceed to let the Service invocation happen. Clearly, this will hamper performance but at least it will raise awareness that a problem exists. Clearly, this could be tied into a more robust notification system, but it is a crude example of the possibilities. The important thing is, your Service remains available while the other services (e.g. Redis) that the application service depends on, may have failed.

In this implementation (vs. my previous explanation) I chose to delegate to the underlying, actual RedisCache implementation to let the Exception occur, then knowing full well a problem with Redis exists, and so that you can deal with the Exception appropriately. However, if you are a certain that the Exception is related to a connection problem upon inspection, you can return "null" to let Spring Caching Infrastructure proceed as if it were a Cache "miss" (i.e. bad Redis Connection == Cache miss, in this case).

I know something like this should help your problem as I built a similar prototype of a "custom" CacheManager implementation for GemFire and one of Pivotal's customers. In that particular UC, the Cache "miss" had to be triggered by an "out-of-date version" of the application domain object where production had a mix of newer and older application clients connecting to GemFire through Spring's Caching Abstraction. The application domain object fields would change in newer versions of the app for instance.

Anyway, hope this helps or gives you more ideas.

Cheers!

like image 150
John Blum Avatar answered Sep 20 '22 05:09

John Blum