Mocking elasticsearch-py calls

Question

I'm writing a CLI to interact with elasticsearch using the elasticsearch-py library. I'm trying to mock elasticsearch-py functions in order to test my functions without calling my real cluster.

I read this question and this one but I still don't understand.

main.py

Escli inherits from cliff's App class

class Escli(App):
    _es = elasticsearch5.Elasticsearch()

settings.py

from escli.main import Escli

class Settings:
    def get(self, sections):
        raise NotImplementedError()

class ClusterSettings(Settings):

    def get(self, setting, persistency='transient'):
        settings = Escli._es.cluster\
                    .get_settings(include_defaults=True, flat_settings=True)\
                    .get(persistency)\
                    .get(setting)

        return settings

settings_test.py

import escli.settings


class TestClusterSettings(TestCase):
    def setUp(self):
        self.patcher = patch('elasticsearch5.Elasticsearch')
        self.MockClass = self.patcher.start()

    def test_get(self):
        # Note this is an empty dict to show my point
        # it will contain childs dict to allow my .get(persistency).get(setting)
        self.MockClass.return_value.cluster.get_settings.return_value = {}

        cluster_settings = escli.settings.ClusterSettings()
        ret = cluster_settings.get('cluster.routing.allocation.node_concurrent_recoveries', persistency='transient')
        # ret should contain a subset of my dict defined above

I want to have Escli._es.cluster.get_settings() to return what I want (a dict object) in order to not make the real HTTP call, but it keeps doing it.

What I know:

• In order to mock an instance method I have to do something like MagicMockObject.return_value.InstanceMethodName.return_value = ...

• I cannot patch Escli._es.cluster.get_settings because Python tries to import Escli as module, which cannot work. So I'm patching the whole lib.

I desperately tried to put some return_value everywhere but I cannot understand why I can't mock that thing properly.

Answer 1

You should be mocking with respect to where you are testing. Based on the example provided, this means that the Escli class you are using in the settings.py module needs to be mocked with respect to settings.py. So, more practically, your patch call would look like this inside setUp instead:

self.patcher = patch('escli.settings.Escli')

With this, you are now mocking what you want in the right place based on how your tests are running.

Furthermore, to add more robustness to your testing, you might want to consider speccing for the Elasticsearch instance you are creating in order to validate that you are in fact calling valid methods that correlate to Elasticsearch. With that in mind, you can do something like this, instead:

self.patcher = patch('escli.settings.Escli', Mock(Elasticsearch))

To read a bit more about what exactly is meant by spec, check the patch section in the documentation.

As a final note, if you are interested in exploring the great world of pytest, there is a pytest-elasticsearch plugin created to assist with this.