Elasticsearch testing(unit/integration) best practices in C# using Nest

Question

I've been seraching for a while how should I test my data access layer with not a lot of success. Let me list my concerns:

1. Unit tests

This guy (here: Using InMemoryConnection to test ElasticSearch) says that:

Although asserting the serialized form of a request matches your expectations in the SUT may be sufficient.

Does it really worth to assert the serialized form of requests? Do these kind of tests have any value? It doesn't seem likely to change a function that should not change the serialized request.

If it does worth it, what is the correct way to assert these reqests?

2. Unit tests once again

Another guy (here: ElasticSearch 2.0 Nest Unit Testing with MOQ) shows a good looking example:

void Main()
{
    var people = new List<Person>
    {
        new Person { Id = 1 },
        new Person { Id = 2 },
    };

    var mockSearchResponse = new Mock<ISearchResponse<Person>>();
    mockSearchResponse.Setup(x => x.Documents).Returns(people);

    var mockElasticClient = new Mock<IElasticClient>();
    mockElasticClient.Setup(x => x
        .Search(It.IsAny<Func<SearchDescriptor<Person>, ISearchRequest>>()))
        .Returns(mockSearchResponse.Object);

    var result = mockElasticClient.Object.Search<Person>(s => s);

    Assert.AreEqual(2, result.Documents.Count()).Dump();
}

public class Person
{
    public int Id { get; set;}
}

Probably I'm missing something but I can't see the point of this code snippet. First he mocks an ISearchResponse to always return the people list. then he mocks an IElasticClient to return this previous search response to any search request he makes.

Well it doesn't really surprise me the assertion is true after that. What is the point of these kind of tests?

3. Integration tests

Integration tests do make more sense to me to test a data access layer. So after a little search i found this (https://www.nuget.org/packages/elasticsearch-inside/) package. If I'm not mistaken this is only about an embedded JVM and an ES. Is it a good practice to use it? Shouldn't I use my already running instance?

If anyone has good experience with testing that I didn't include I would happily hear those as well.

Answer 1

Each of the approaches that you have listed may be a reasonable approach to take, depending on exactly what it is you are trying to achieve with your tests. you haven't specified this in your question :)

Let's go over the options that you have listed

1. Asserting the serialized form of the request to Elasticsearch may be a sufficient approach if you build a request to Elasticsearch based on a varying number of inputs. You may have tests that provide different input instances and assert the form of the query that will be sent to Elasticsearch for each. These kinds of tests are going to be fast to execute but make the assumption that the query that is generated and you are asserting the form of is going to return the results that you expect.
2. This is another form of unit test that stubs out the interaction with the Elasticsearch client. The system under test (SUT) in this example is not the client but another component that internally uses the client, so the interaction with the client is controlled through the stub object to return an expected response. The example is contrived in that in a real test, you wouldn't assert on the results of the client call as you point out but rather on the output of the SUT.
3. Integration/Behavioural tests against a known data set within an Elasticsearch cluster may provide the most value and go beyond points 1 and 2 as they will not only incidentally test the generated queries sent to Elasticsearch for a given input, but will also be testing the interaction and producing an expected result. No doubt however that these types of test are harder to setup than 1 and 2, but the investment in setup may be outweighed by their benefit for your project.

So, you need to ask yourself what kinds of tests are sufficient to achieve the level of assurance that you require to assert that your system is doing what you expect it to do; it may be a combination of all three different approaches for different elements of the system.

You may want to check out how the .NET client itself is tested; there are components within the Tests project that spin up an Elasticsearch cluster with different plugins installed, seed it with known generated data and make assertions on the results. The source is open and licensed under Apache 2.0 license, so feel free to use elements within your project :)