Embedded Postgres for Spring Boot Tests

Question

I'm building a Spring Boot app, backed by Postgres, using Flyway for database migrations. I've been bumping up against issues where I cannot produce a migration that generates the desired outcome in both Postgres, and the embedded unit test database (even with Postgres compatibility mode enabled). So I am looking at using embedded Postgres for unit tests.

I came across an embedded postgres implementation that looks promising, but don't really see how to set it up to run within Spring Boot's unit test framework only (for testing Spring Data repositories). How would one set this up using the mentioned tool or an alternative embedded version of Postgres?

Answer 1

I'm the author of the embedded-database-spring-test library that was mentioned by @MartinVolejnik. I think the library should meet all your needs (PostgreSQL + Spring Boot + Flyway + integration testing). I'm really sorry that you're having some trouble, so I've created a simple demo app that demonstrates the use of the library together with Spring Boot framework. Below I summarized some basic steps that you need to do.

Maven configuration

Add the following maven dependency:

<dependency>
    <groupId>io.zonky.test</groupId>
    <artifactId>embedded-database-spring-test</artifactId>
    <version>2.0.1</version>
    <scope>test</scope>
</dependency>

Flyway configuration

Add the following property to your application configuration:

# Sets the schemas managed by Flyway -> change the xxx value to the name of your schema
# flyway.schemas=xxx // for spring boot 1.x.x
spring.flyway.schemas=xxx // for spring boot 2.x.x

Further, make sure that you do not use org.flywaydb.test.junit.FlywayTestExecutionListener. Because the library has its own test execution listener that can optimize database initialization and this optimization has no effect if the FlywayTestExecutionListener is applied.

Example

An example of test class demonstrating the use of the embedded database:

@RunWith(SpringRunner.class)
@DataJpaTest
@AutoConfigureEmbeddedDatabase
public class SpringDataJpaAnnotationTest {

    @Autowired
    private PersonRepository personRepository;

    @Test
    public void testEmbeddedDatabase() {
        Optional<Person> personOptional = personRepository.findById(1L);

        assertThat(personOptional).hasValueSatisfying(person -> {
            assertThat(person.getId()).isNotNull();
            assertThat(person.getFirstName()).isEqualTo("Dave");
            assertThat(person.getLastName()).isEqualTo("Syer");
        });
    }
}

Answer 2

Another quite clean solution to that problem is to use the TestContainers library. The only caveat is that it requires Docker.

Integration Test:

@RunWith(SpringRunner.class)
@SpringBootTest
@ContextConfiguration(initializers = {ApplicationTestsIT.Initializer.class})
public class ApplicationTestsIT {

    private static int POSTGRES_PORT = 5432;

    @Autowired
    private FooRepository fooRepository;

    @ClassRule
    public static PostgreSQLContainer postgres = new PostgreSQLContainer<>("postgres")
            .withDatabaseName("foo")
            .withUsername("it_user")
            .withPassword("it_pass")
            .withInitScript("sql/init_postgres.sql");

    static class Initializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
        public void initialize(ConfigurableApplicationContext configurableApplicationContext) {
            TestPropertyValues.of(
                    "spring.data.postgres.host=" + postgres.getContainerIpAddress(),
                    "spring.data.postgres.port=" + postgres.getMappedPort(POSTGRES_PORT),
                    "spring.data.postgres.username=" + postgres.getUsername(),
                    "spring.data.postgres.password=" + postgres.getPassword()
            ).applyTo(configurableApplicationContext.getEnvironment());
        }
    }

    @Test
    public void fooRepositoryTestIT() {
        ...
    }

Dependency configuration:
pom.xml:

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>postgresql</artifactId>
    <scope>test</scope>
</dependency>

build.gradle:

testCompile "org.testcontainers:postgresql:x.x.x"

Links:
TestContainers - Databases
TestContainers - Postgres Module

Answer 3

The configuration below works well with Spring Boot 2.0.

The advantage over embedded-database-spring-test is that this solution doesn't push Flyway into the classpath, possibly messing up Spring Boot's autoconfiguration.

@Configuration
@Slf4j
public class EmbeddedPostgresConfiguration {

    @Bean(destroyMethod = "stop")
    public PostgresProcess postgresProcess() throws IOException {
        log.info("Starting embedded Postgres");

        String tempDir = System.getProperty("java.io.tmpdir");
        String dataDir = tempDir + "/database_for_tests";
        String binariesDir = System.getProperty("java.io.tmpdir") + "/postgres_binaries";

        PostgresConfig postgresConfig = new PostgresConfig(
                Version.V10_3,
                new AbstractPostgresConfig.Net("localhost", Network.getFreeServerPort()),
                new AbstractPostgresConfig.Storage("database_for_tests", dataDir),
                new AbstractPostgresConfig.Timeout(60_000),
                new AbstractPostgresConfig.Credentials("bob", "ninja")
        );

        PostgresStarter<PostgresExecutable, PostgresProcess> runtime =
                PostgresStarter.getInstance(EmbeddedPostgres.cachedRuntimeConfig(Paths.get(binariesDir)));
        PostgresExecutable exec = runtime.prepare(postgresConfig);
        PostgresProcess process = exec.start();

        return process;
    }

    @Bean(destroyMethod = "close")
    @DependsOn("postgresProcess")
    DataSource dataSource(PostgresProcess postgresProcess) {
        PostgresConfig postgresConfig = postgresProcess.getConfig();

        val config = new HikariConfig();
        config.setUsername(postgresConfig.credentials().username());
        config.setPassword(postgresConfig.credentials().password());
        config.setJdbcUrl("jdbc:postgresql://localhost:" + postgresConfig.net().port() + "/" + postgresConfig.storage().dbName());

        return new HikariDataSource(config);
    }
}

Maven:

        <dependency>
            <groupId>ru.yandex.qatools.embed</groupId>
            <artifactId>postgresql-embedded</artifactId>
            <version>2.9</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
        </dependency>

The class is based on the code I found here: https://github.com/nkoder/postgresql-embedded-example

I modified it to use HikariDatasource (Spring Boot's default) for proper connection pooling. The binariesDir and dataDir are used to avoid costly extraction+initdb in repeated tests.

Answer 4

Take a look at this: https://github.com/zonkyio/embedded-database-spring-test. Just to be clear, it's meant for integration testing. Meaning the Spring context is initialised during the individual test.

As per the tools documentation, all you need to do is to place @AutoConfigureEmbeddedDatabase annotation above class:

@RunWith(SpringRunner.class)
@AutoConfigureEmbeddedDatabase
@ContextConfiguration("/path/to/app-config.xml")
public class FlywayMigrationIntegrationTest {

    @Test
    @FlywayTest(locationsForMigrate = "test/db/migration")
    public void testMethod() {
        // method body...
    }
}

and add Maven dependency:

<dependency>
  <groupId>io.zonky.test</groupId>
  <artifactId>embedded-database-spring-test</artifactId>
  <version>1.1.0</version>
  <scope>test</scope>
</dependency>

To use it together with @DataJpaTest you need to disable the default test database by using the annotation @AutoConfigureTestDatabase(replace = NONE):

@RunWith(SpringRunner.class)
@AutoConfigureTestDatabase(replace = NONE)
@AutoConfigureEmbeddedDatabase
@DataJpaTest
public class SpringDataJpaTest {
// class body...
}

To make the use more comfortable you could also create a composite annotation, something like:

@Documented
@Inherited
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@AutoConfigureTestDatabase(replace = NONE)
@AutoConfigureEmbeddedDatabase
@DataJpaTest
public @interface PostgresDataJpaTest {
}

..and then use it above your test class:

@RunWith(SpringRunner.class)
@PostgresDataJpaTest // custom composite annotation
public class SpringDataJpaTest {
// class body...
}