Is it best practice to test my Web API controllers directly or through an HTTP client?

Question

I'm adding some unit tests for my ASP.NET Core Web API, and I'm wondering whether to unit test the controllers directly or through an HTTP client. Directly would look roughly like this:

[TestMethod]
public async Task GetGroups_Succeeds()
{
    var controller = new GroupsController(
        _groupsLoggerMock.Object,
        _uowRunnerMock.Object,
        _repoFactoryMock.Object
    );

    var groups = await controller.GetGroups();

    Assert.IsNotNull(groups);
}

... whereas through an HTTP client would look roughly like this:

[TestMethod]
public void GetGroups_Succeeds()
{
    HttpClient.Execute();

    dynamic obj = JsonConvert.DeserializeObject<dynamic>(HttpClient.ResponseContent);
    Assert.AreEqual(200, HttpClient.ResponseStatusCode);
    Assert.AreEqual("OK", HttpClient.ResponseStatusMsg);
    string groupid = obj[0].id;
    string name = obj[0].name;
    string usercount = obj[0].userCount;
    string participantsjson = obj[0].participantsJson;
    Assert.IsNotNull(name);
    Assert.IsNotNull(usercount);
    Assert.IsNotNull(participantsjson);
}

Searching online, it looks like both ways of testing an API seem to be used, but I'm wondering what the best practice is. The second method seems a bit better because it naively tests the actual JSON response from the Web API without knowing the actual response object type, but it's more difficult to inject mock repositories this way - the tests would have to connect to a separate local Web API server that itself was somehow configured to use mock objects... I think?

Answer 1

Edit: TL;DR

The conclusion you should do both because each test serves a different purpose.

Answer:

This is a good question, one I often ask myself.

First, you must look at the purpose of a unit test and the purpose of an integration test.

Unit Test :

Unit tests involve testing a part of an app in isolation from its infrastructure and dependencies. When unit testing controller logic, only the contents of a single action are tested, not the behaviour of its dependencies or of the framework itself.

• Things like filters, routing, and model binding will not work.

Integration Test :

Integration tests ensure that an app's components function correctly at a level that includes the app's supporting infrastructures, such as the database, file system, and network. ASP.NET Core supports integration tests using a unit test framework with a test web host and an in-memory test server.

• Things like filters, routing, and model binding will work.

“Best practice” should be thought of as “Has value and makes sense”.

You should ask yourself Is there any value in writing the test, or am I just creating this test for the sake of writing a test?

Let's say your GetGroups() method looks like this.

[HttpGet]
[Authorize]
public async Task<ActionResult<Group>> GetGroups()
{            
    var groups  = await _repository.ListAllAsync();
    return Ok(groups);
}

There is no value in writing a unit test for it! because what you are doing is testing a mocked implementation of _repository! So what is the point of that?! The method has no logic and the repository is only going to be exactly what you mocked it to be, nothing in the method suggests otherwise.

The Repository will have its own set of separate unit tests where you will cover the implementation of the repository methods.

Now let's say your GetGroups() method is more than just a wrapper for the _repository and has some logic in it.

[HttpGet]
[Authorize]
public async Task<ActionResult<Group>> GetGroups()
{            
   List<Group> groups;
   if (HttpContext.User.IsInRole("Admin"))
      groups = await _repository.FindByExpressionAsync(g => g.IsAdminGroup == true);
   else
      groups = await _repository.FindByExpressionAsync(g => g.IsAdminGroup == false);

    //maybe some other logic that could determine a response with a different outcome...
    
    return Ok(groups);
}

Now there is value in writing a unit test for the GetGroups() method because the outcome could change depending on the mocked HttpContext.User value.

Attributes like [Authorize] or [ServiceFilter(….)] will not be triggered in a unit test.

.

Writing integration tests is almost always worth it because you want to test what the process will do when it forms part of an actual application/system/process.

Ask yourself, is this being used by the application/system? If yes, write an integration test because the outcome depends on a combination of circumstances and criteria.

Now even if your GetGroups() method is just a wrapper like in the first implementation, the _repository will point to an actual datastore, nothing is mocked!

So now, not only does the test cover the fact that the datastore has data (or not), it also relies on an actual connection being made, HttpContext being set up properly and whether serialisation of the information works as expected.

Things like filters, routing, and model binding will also work. So if you had an attribute on your GetGroups() method, for example [Authorize] or [ServiceFilter(….)], it will be triggered as expected.

I use xUnit for testing so for a unit test on a controller I use this.

Controller Unit Test:

public class MyEntityControllerShould
{
    private MyEntityController InitializeController(AppDbContext appDbContext)
    {
        var _controller = new MyEntityController (null, new MyEntityRepository(appDbContext));            
        var httpContext = new DefaultHttpContext();
        var context = new ControllerContext(new ActionContext(httpContext, new RouteData(), new ActionDescriptor()));
        _controller.ControllerContext = context;
        return _controller;
    }

    [Fact]
    public async Task Get_All_MyEntity_Records()
    {
      // Arrange
      var _AppDbContext = AppDbContextMocker.GetAppDbContext(nameof(Get_All_MeetUp_Records));
      var _controller = InitializeController(_AppDbContext);
    
     //Act
     var all = await _controller.GetAllValidEntities();
     
     //Assert
     Assert.True(all.Value.Count() > 0);
    
     //clean up otherwise the other test will complain about key tracking.
     await _AppDbContext.DisposeAsync();
    }
}

The Context mocker used for unit testing.

public class AppDbContextMocker
{
    /// <summary>
    /// Get an In memory version of the app db context with some seeded data
    /// </summary>
    /// <param name="dbName"></param>
    /// <returns></returns>
    public static AppDbContext GetAppDbContext(string dbName)
    {
        //set up the options to use for this dbcontext
        var options = new DbContextOptionsBuilder<AppDbContext>()
            .UseInMemoryDatabase(dbName)                
            .Options;
        var dbContext = new AppDbContext(options);
        dbContext.SeedAppDbContext();
        return dbContext;
    }
}

The Seed extension.

public static class AppDbContextExtensions
{
   public static void SeedAppDbContext(this AppDbContext appDbContext)
   {
       var myEnt = new MyEntity()
       {
          Id = 1,
          SomeValue = "ABCD",
       }
       appDbContext.MyENtities.Add(myEnt);
       //add more seed records etc....

        appDbContext.SaveChanges();
        //detach everything
        foreach (var entity in appDbContext.ChangeTracker.Entries())
        {
           entity.State = EntityState.Detached;
        }
    }
}

and for Integration Testing: (this is some code from a tutorial, but I can't remember where I saw it, either youtube or Pluralsight)

setup for the TestFixture

public class TestFixture<TStatup> : IDisposable
{
    /// <summary>
    /// Get the application project path where the startup assembly lives
    /// </summary>    
    string GetProjectPath(string projectRelativePath, Assembly startupAssembly)
    {
        var projectName = startupAssembly.GetName().Name;

        var applicationBaseBath = AppContext.BaseDirectory;

        var directoryInfo = new DirectoryInfo(applicationBaseBath);

        do
        {
            directoryInfo = directoryInfo.Parent;
            var projectDirectoryInfo = new DirectoryInfo(Path.Combine(directoryInfo.FullName, projectRelativePath));
            if (projectDirectoryInfo.Exists)
            {
                if (new FileInfo(Path.Combine(projectDirectoryInfo.FullName, projectName, $"{projectName}.csproj")).Exists)
                    return Path.Combine(projectDirectoryInfo.FullName, projectName);
            }
        } while (directoryInfo.Parent != null);

        throw new Exception($"Project root could not be located using application root {applicationBaseBath}");
    }

    /// <summary>
    /// The temporary test server that will be used to host the controllers
    /// </summary>
    private TestServer _server;

    /// <summary>
    /// The client used to send information to the service host server
    /// </summary>
    public HttpClient HttpClient { get; }

    public TestFixture() : this(Path.Combine(""))
    { }

    protected TestFixture(string relativeTargetProjectParentDirectory)
    {
        var startupAssembly = typeof(TStatup).GetTypeInfo().Assembly;
        var contentRoot = GetProjectPath(relativeTargetProjectParentDirectory, startupAssembly);

        var configurationBuilder = new ConfigurationBuilder()
            .SetBasePath(contentRoot)
            .AddJsonFile("appsettings.json")
            .AddJsonFile("appsettings.Development.json");


        var webHostBuilder = new WebHostBuilder()
            .UseContentRoot(contentRoot)
            .ConfigureServices(InitializeServices)
            .UseConfiguration(configurationBuilder.Build())
            .UseEnvironment("Development")
            .UseStartup(typeof(TStatup));

        //create test instance of the server
        _server = new TestServer(webHostBuilder);

        //configure client
        HttpClient = _server.CreateClient();
        HttpClient.BaseAddress = new Uri("http://localhost:5005");
        HttpClient.DefaultRequestHeaders.Accept.Clear();
        HttpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

    }

    /// <summary>
    /// Initialize the services so that it matches the services used in the main API project
    /// </summary>
    protected virtual void InitializeServices(IServiceCollection services)
    {
        var startupAsembly = typeof(TStatup).GetTypeInfo().Assembly;
        var manager = new ApplicationPartManager
        {
            ApplicationParts = {
                new AssemblyPart(startupAsembly)
            },
            FeatureProviders = {
                new ControllerFeatureProvider()
            }
        };
        services.AddSingleton(manager);
    }

    /// <summary>
    /// Dispose the Client and the Server
    /// </summary>
    public void Dispose()
    {
        HttpClient.Dispose();
        _server.Dispose();
        _ctx.Dispose();
    }

    AppDbContext _ctx = null;
    public void SeedDataToContext()
    {
        if (_ctx == null)
        {
            _ctx = _server.Services.GetService<AppDbContext>();
            if (_ctx != null)
                _ctx.SeedAppDbContext();
        }
    }
}

and use it like this in the integration test.

public class MyEntityControllerShould : IClassFixture<TestFixture<MyEntityApp.Api.Startup>>
{
    private HttpClient _HttpClient;
    private const string _BaseRequestUri = "/api/myentities";

    public MyEntityControllerShould(TestFixture<MyEntityApp.Api.Startup> fixture)
    {
        _HttpClient = fixture.HttpClient;
        fixture.SeedDataToContext();
    }

    [Fact]
    public async Task Get_GetAllValidEntities()
    {
        //arrange
        var request = _BaseRequestUri;

        //act
        var response = await _HttpClient.GetAsync(request);

        //assert
        response.EnsureSuccessStatusCode(); //if exception is not thrown all is good

        //convert the response content to expected result and test response
        var result = await ContentHelper.ContentTo<IEnumerable<MyEntities>>(response.Content);
        Assert.NotNull(result);
    }
}

Added Edit: In conclusion, you should do both, because each test serves a different purpose.

Looking at the other answers you will see that the consensus is to do both.

Answer 2

TL;DR

Is it best practice to test [...] directly or through an HTTP client?

Not "or" but "and". If you serious about best practices of testing - you need both tests.

---

First test is a unit test. But the second one is an integration test.

There is a common consensus (test pyramid) that you need more unit tests comparing to the number of integration tests. But you need both.

There are many reasons why you should prefer unit tests over integration tests, most of them boil down to the fact that unit test are small (in all senses) and integration tests - aren't. But the main 4 are:

1. Locality

   When your unit test fails, usually, just from it's name you can figure out the place where the bug is. When integration test becomes red, you can't say right away where is the issue. Maybe it's in the controller.GetGroups or it's in the HttpClient, or there is some issue with the network.

   Also, when you introduce a bug in your code it's quite possible that only one of unit tests will become red, while with integration tests there are more chances that more than one of them will fail.

2. Stability

   With a small project which you can test on you local box you probably won't notice it. But on a big project with distributed infrastructure you will see blinking tests all the time. And that will become a problem. At some point you can find yourself not trusting test results anymore.

3. Speed

   With a small project with a small number of tests you won't notice it. But on a bit project it will become a problem. (Network delays, IO delays, initialization, cleanup, etc., etc.)

4. Simplicity

   You've noticed it yourself.

   But that not always true. If you code is poorly structured, then it's easier to write integration tests. And that's one more reason why you should prefer unit tests. In some way they force you to write more modular code (and I'm not taking about Dependency Injection).

But also keep in mind that best practices are almost always about big projects. If your project is small, and will stay small, there are a big chance that you'll be better off with strictly opposite decisions.

Write more tests. (Again, that means - both). Become better at writing tests. Delete them latter.

Practice makes perfect.