Catch ConnectionError with Django-Haystack and ElasticSearch

Question

I'm currently using django-haystack and elasticsearch in a project and all works as expected when elasticsearch is running.

Haystack Settings:

HAYSTACK_CONNECTIONS = {
'default': {
    'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
    'URL': 'http://127.0.0.1:9200/',
    'INDEX_NAME': 'haystack',
},
}

HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'

I'm using RealtimeSignalProcessor for a realtime index update.

The problem comes when elasticsearch is down, because trying to add/update any object gives us the following error:

ConnectionError(('Connection aborted.', error(111, 'Connection refused'))) caused by: ProtocolError(('Connection aborted.', error(111, 'Connection refused')))

Is there a way of catch/manage that error?

It would be useful in production environment in order to allow users to add/update objects without crashing, when elasticsearch is down.

Thanks in advance.

Answer 1

I suggest you subclass ElasticSearchBackend and wrap the update, remove and clear methods around a decorator that captures the exceptions. That way you keep elasticsearch features, but you are able to override the behaviour of them.

I use to wrap them with a decorator, a mute-error one:

def mute_error(f):      
    def error_wrapper(*args, **kwargs):  
        try:  
            return f(*args, **kwargs)  
        except:
            print('Connection Error')    

    return error_wrapper

Then add it to your project, configure the HAYSTACK_BACKEND:

HAYSTACK_CONNECTIONS = {
    'default': {
        'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
        'URL': 'http://127.0.0.1:9200/',
        'INDEX_NAME': 'haystack',
    },
    'robust_elasticsearch':{
        'ENGINE': 'YOURAPP.backend.RobustElasticSearchEngine',
        'URL': 'http://127.0.0.1:9200/',
        'INDEX_NAME': 'haystack',
    }
}

Have a look at django-haystack documentation. You should also create a subclass of BaseEngine, this is not properly documented.

Here it is the code:

from django.utils.decorators import method_decorator
from haystack.backends.elasticsearch_backend import ElasticsearchSearchBackend, ElasticsearchSearchEngine
from haystack.backends import BaseEngine
from haystack.backends import log_query
from urllib3.exceptions import ProtocolError, ConnectionError


class RobustElasticSearchBackend(ElasticsearchSearchBackend):
    """A robust backend that doesn't crash when no connection is available"""

    def mute_error(f):

        def error_wrapper(self, *args, **kwargs):
            try:
                return f(self, *args, **kwargs)
            except TransportError:
                self.log.warn('Connection Error: elasticsearch communication error') 
        return error_wrapper

    def __init__(self, connectionalias, **options):
        super(RobustElasticSearchBackend, self).__init__(connectionalias, **options)

    @mute_error
    def update(self, indexer, iterable, commit=True):
        super(RobustElasticSearchBackend, self).update(indexer, iterable, commit)

    @mute_error
    def remove(self, obj, commit=True):
        super(RobustElasticSearchBackend, self).remove(obj, commit)

    @mute_error
    def clear(self, models=[], commit=True):
        super(RobustElasticSearchBackend, self).clear(models, commit)

class RobustElasticSearchEngine(ElasticsearchSearchEngine):
    backend = RobustElasticSearchBackend

We are just overriding the engine, and not the SearchQuery subclass, since the one provided by elasticsearch class by default is enough for us, by now.

Answer 2

To make this work for my setup I had to

import elasticsearch

and edit to:

def mute_error(f):

    def error_wrapper(self, *args, **kwargs):
        try:
            return f(self, *args, **kwargs)
        except elasticsearch.TransportError:
            self.log.warn('Connection Error: elasticsearch communication error')
    return error_wrapper