Integration Tests With WebApplicationFactory and Test Containers

My team are pretty good at building integration tests, we use app.settings per environment, powershell and teamcity. Its fairly complicated but works really well. We dont always use WebApplicationFactory to bring up our API’s in memory as its not always sensible to do so.

I was inspired by a video from Nick Chapsas titled The cleanest way to use Docker for testing in .NET to investigate the nuget package Test Containers | Testcontainers 2.1.0.

Checkout Nick’s From Zero to Hero: Integration testing in ASP.NET Core course - amazing content brother! - Keep coding!

Problems faced with integration test

Testcontainers addresses a few problems faced with integration tests

  • Parallelize integration test execution - Race conditions (collisions) where some tests fail because of orphaned data from another test. Example: A SelectAll may expect 5 records in the database however a preceding Insert may have changed this number to 6 so the SelectAll will fail its validation as 6 > 5

  • Setup and teardown of the database - Its overhead to have to worry about setting up infastructure outside of the code, another powershell or teamcity step which is just a possible point of failure.

  • Create a database specific for test execution - this will be in isolation, so for each test we will create a database using the Testcontainers mechanism.

Integration Tests

The samples assume you already have some integration tests setup, this is a simple CRUD API with integration tests over the controllers. My source code is based on Nicks video and can be downloaded at https://github.com/carlpaton/IntegrationTestsWithTestContainers.

My test classes are shown below. The problem with integration tests are they dont really customize the core project. Nick describes this as then calling the real database, the problem with this approach is we have test data remnants which we could delete after each test execution but that implies we need to have a database running at all times.

Other systems might be using this so we will run into problems with everyone running this locally and when we try do continuous integration.

The current integration tests are:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
CreateCustomerControllerTests
 - Create_CreatesUser_WhenDataIsValid
 - Create_ReturnsValidationError_WhenDataIsInvalid

DeleteCustomerControllerTests
 - Delete_DeletesUserByCustomerId_WhenCustomerExists

GetAllCustomerControllerTests
 - GetAll_ReturnsAllCustomers_WhenTheyExist

GetCustomerControllerTests
 - Read_ReturnsCustomerById_WhenItExists

PingControllerTests
 - Ping_ReturnsOk

UpdateCustomerControllerTests
 - Update_UpdatesTheGivenCustomer_WhenParametersAreValid

Example Code

TestBase

I used TestBase to expose a HttpClient which is created using CustomerApiFactory -> WebApplicationFactory using the method CreateClient.

All tests then inherit TestBase and use dependency injection to instanciate the CustomerApiFactory using IClassFixture<CustomerApiFactory> which is used to indicate a test has per-test-class fixture data.

1
2
3
4
public class CreateCustomerControllerTests : TestBase
{
    public CreateCustomerControllerTests(CustomerApiFactory factory) : base(factory) { }
...

CustomerApiFactory

  1. To get started install the nuget package Testcontainers 2.1.0

  2. I use WebApplicationFactory and the class CustomerApiFactory which would spin up an in-memory version of the CustomerApi.

  3. IAsyncLifetime is then used to manage the life time of PostgreSqlTestcontainer. This starts the container in the beginning of the tests and then stops them at the end of the tests using xUnits IAsyncLifetime interface which provides the methods InitializeAsync and DisposeAsync which will run per test execution. As we have this in a collection it will run per class that has tests in it.

This is a great compromise between isolation of tests as its only limited to the collection.

  1. I then use override ConfigureWebHost to remove the DatabaseContext instance and replace it with our test instance, which uses the ConnectionString from PostgreSqlTestcontainer.

The value of this is the port is always unique, this is what the cool kids call an ephemeral container (a special type of container that runs temporarily in an existing Pod to accomplish user-initiated actions). So each test class gets a brand new and clean instance of a postgres database.

Note that we use ConfigureTestServices instead of ConfigureServices.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
public class CustomerApiFactory : WebApplicationFactory<Program>, IAsyncLifetime
{
    private readonly PostgreSqlTestcontainer _dbContainer;
    private readonly string _appSettingsFile = "appsettings.Test.json";
    private readonly IConfiguration _configuration;

    public CustomerApiFactory()
    {
        _configuration = new ConfigurationBuilder()
            .SetBasePath(Directory.GetParent(AppContext.BaseDirectory).FullName)
            .AddJsonFile(_appSettingsFile)
            .Build();

        _dbContainer = new TestcontainersBuilder<PostgreSqlTestcontainer>()
            .WithDatabase(new PostgreSqlTestcontainerConfiguration
            {
                Database = _configuration.GetSection("PostgreSQL:Database").Get<string>(),
                Username = _configuration.GetSection("PostgreSQL:Username").Get<string>(),
                Password = _configuration.GetSection("PostgreSQL:Password").Get<string>(),
            })
            .Build();
    }

    public Task InitializeAsync()
    {
        // start the container per collection
        return _dbContainer.StartAsync();
    }

    // we use the new keyword to avoid conflicts with IAsyncDisposable Interface - https://docs.microsoft.com/en-us/dotnet/api/system.iasyncdisposable
    public new Task DisposeAsync()
    {
        // stop the container per collection
        return _dbContainer.StopAsync();
    }

    // we override ConfigureWebHost
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        var maxRetryCount = _configuration.GetSection("PostgreSQL:MaxRetryCount").Get<int>();
        var maxRetryDelay = _configuration.GetSection("PostgreSQL:MaxRetryDelay").Get<int>();

        builder
        .UseEnvironment("Test")
        .UseContentRoot(Directory.GetCurrentDirectory())
        .ConfigureAppConfiguration(config => config.AddJsonFile(_appSettingsFile))
        .ConfigureTestServices(services =>
        {
             // We dont really have to do this step, if you register another `DatabaseContext` ontop of this and through DI only resolve 1 you will get the latest
            // Its safer to remove so you know only one instance exists in the DI container
            var descriptor = services.SingleOrDefault(d => d.ServiceType == typeof(DbContextOptions<DatabaseContext>));
            services.Remove(descriptor);

            // Replace with test instance
            services.AddDbContext<DatabaseContext>((serviceProvider, optionsBuilder) =>
            {
                optionsBuilder.UseNpgsql(
                    _dbContainer.ConnectionString,
                    npgsqlOptionsAction =>
                    {
                        npgsqlOptionsAction.EnableRetryOnFailure(
                            maxRetryCount,
                            TimeSpan.FromSeconds(maxRetryDelay),
                            null);
                    });
            });
        });
    }
}

  1. Running the tests now will spin up a container for postgres just for the test collection. This means we can seed any test data we may want during startup and we dont need to worry about clearning it out as the containers are ephemeral.

  2. The tests can now run and their container will automagically be spun up <3. Notice each have their own port and cannot be polluted with another tests data. Additionally the containers are deleted by the framework. The is the most hard out nerd thing I’ve seen in a while :D

Under the hood the container orchestration is done by testcontainers / moby-ryuk

throw-away-containers-per-test

  1. Currently there is support for other databases listed below. For ones that dont exist its possible to use 3rd party packages ontop using the Fluent API described below.

See https://github.com/testcontainers/testcontainers-dotnet#pre-configured-containers for the latest list.

1
2
3
4
5
6
7
CouchbaseTestconnectionConfiguration
OracleTestconnectionConfiguration
RedisTestconnectionConfiguration
CouchDbTestconnectionConfiguration
MongoDbTestconnectionConfiguration
MsSqlTestconnectionConfiguration
MySqlTestconnectionConfiguration

Test Container Fluent API

The test container framework has a fluent API that allows us to load up any docker image that is not natively supported, this is an example with drawbacks such as static port.

All of the commands are listed here - https://github.com/testcontainers/testcontainers-dotnet#supported-commands

1
2
3
4
5
6
7
8
9
10
// private member for test containers using fluent api to construct the builder
private readonly TestcontainersContainer _dbContainer = 
    new TestcontainersBuilder<TestcontainersContainer>()
        .WithImage("postgres:14.4") // https://hub.docker.com/_/postgres/
        .WithEnvironment("POSTGRE_USER", "postgres")
        .WithEnvironment("POSTGRE_PASSWORD", "password")
        .WithEnvironment("POSTGRE_DB", "mydb")
        .WithPortBiding(5555, 5432) // 5555 external -> 5432 internal
        .WithWaitStrategy(Wait.ForUnixContainer().UntilPortIsAvailable(5432)) // tells the code to not proceed with anything without having this OK. Example the container is up and running
        .Build();

References