Integration tests using databases in Scala

If you use a SQL or NoSQL database, you want to have an integration test in place that tests the code that uses the database. One way to do this, is to require that an actual database is running, but this requires a database to be running on your build server and you need to be sure that the database is in the desired state before each test. Furthermore, if multiple builds run simultaneously using the same database, the results are unpredictable.

Sometimes the in memory SQL database H2 is used for tests, configured in such a way that it mimics the behavior of the actual database. For NoSQL databases this can’t be done and for SQL databases this is not a good idea because you’re never entirely sure that H2 will exactly mimic the behavior so testing with an actual database is still necessary.

The easiest way to create integration tests using a database is having the test start a Docker container running the database. Starting and running the Docker is lightweight and can be done on the build server. Furthermore, having a new container for every test run guarantees that the database is in a known state before the test starts.

There are a few libraries to make starting and stopping a Docker container before and after your test easier. A well known one is Testcontainers. For this library there is also the Scala wrapper Testcontainers-scala available.

Using Testcontainers-scala

If you want to use Testcontainers-scala for your Scala integration tests, add the following library to your build.sbt:

"com.dimafeng" %% "testcontainers-scala" % TestcontainersVersion % "test"

Using ScalaTest you can now extend ForAllTestContainer if you want the container to be started before all tests and stopped after all tests or ForEachTestContainer if you want the container to be started and stopped for each test.

You need to override the val container to provide the container to start. E.g. for a test using PostgreSQL:

class MySpec extends FlatSpec with ForAllTestContainer {
 override val container = PostgreSQLContainer()
...
}

There are couple of containers defined for frequently used SQL databases.

To connect to PostgreSQL in a test, you use container.jdbcUrl. This will return a JDBC URL with the correct host and port to connect to the Docker container running PostgreSQL.  Use container.username and container.password for the username and password of the database.

Note: these values are only available after the container has been started, so use a lazy val if you define a class variable using these values.

Using a generic container

If you use a database for which there is no standard container class available, you need to use the GenericContainer class to define your own. GenericContainer.apply has several parameters you can set. The most important ones are the imageName, exposedPorts, and waitStrategy.

imageName obviously defines the docker image to use. exposedPorts is a Seq[Int] of internal ports that are mapped and exposed to outside the container and waitStrategy defines how to wait until the container has fully started before running the tests.

There are different wait strategies available and you could also implement your own, but one of the available ones will probably suit your needs. The default strategy is HostPortWaitStrategy, which waits for exposed ports to be available.

Another one is the HttpWaitStrategy which waits for a path to return a defined status code.

Finally, there is LogMessageWaitStrategy, which waits for a message to appear in the output.

As an example, we’re looking at how to define a container for Aerospike, a key/value NoSQL database. Aerospike listens by default on port 3000 for connections. Using the HostPortWaitStrategy, however doesn’t work because if Aerospike is already listening on this port, it hasn’t fully started yet, so connecting at that time would result in errors. We use the LogMessageWaitStrategy instead to wait for the log message migrations: complete to appear in the output.

So we create the Aerospike container in our test class with:

override val container = GenericContainer(
 imageName = "aerospike:latest",
 exposedPorts = Seq(3000),
 waitStrategy = new LogMessageWaitStrategy().withRegEx(".*migrations: complete.*\\s"))

Using container.containerIpAddress we get the host and with container.mappedPort(3000) we get the port. Using the host and port we can now create a connection to the Aerospike container.


Posted

in

by

Tags:

Comments

3 responses to “Integration tests using databases in Scala”

  1. Hoda Avatar
    Hoda

    Thanks for knowledge sharing Jasper!

  2. Abdullah Odibat Avatar
    Abdullah Odibat

    Thanks. how much time did the docker take to start?

  3. Jasper Avatar
    Jasper

    How much time it takes for the container to start depends. For the Aerospike container the start is slower than for PostgreSQL. Stopping and starting the container for every single test will usually add too much time to the test phase of the build, so usually it is better to start it once before several tests.

Leave a Reply

Your email address will not be published. Required fields are marked *