Concourse CI - how to run functional tests?

Question

we are in the middle of process migrating from Jenkins to Concourse CI and everything was pretty smooth so far. But now I have the issue, that I don't know how to solve. I would like to get any advices from the community.

What I am trying to do is a job that can run integrational or functional (web) tests using Selenium. There are few issues for us:

1. To run web tests I need to set up the database (and optionally, the searching engine, proxy and etc...) proxy to imitate the production environment as close at possible. Ideally, it should be set up by docker-compose.
2. This database service should run in parallel of my tests
3. This database service should not return anything, neither error or success, because it only starts the database and nothing else
4. My web-tests should not be started until the database is ready
5. This database service should be stopped when all the web-tests were finished

As you can see, it's pretty non-trivial task. Of course, I can create an big uber-container that contains everything I need, but this is bad solution. Another option is to create a shell-script for that, but this is not flexible enough.

Is there any example how I could implement that or good practices for this issue?

Thanks!

Answer 1

Since version 1.3.0 it appears you can run Docker-compose in a task: https://github.com/concourse/concourse/issues/324

This appears to work:

jobs:
  - name: docker-compose
    public: true
    serial: true
    plan:
      - do:
        - task: docker-compose
          timeout: 20m
          privileged: true
          config:
            platform: linux
            image_resource:
              type: docker-image
              source: {repository: "mumoshu/dcind", tag: "latest"}
            run:
              path: sh
              args:
                - -exc
                - |
                  source /docker-lib.sh
                  start_docker
                  docker ps
                  docker-compose version

Answer 2

This is comment from author of Concourse:

There is no Docker binary or socket on the host - they're just running a Garden backend (probably Guardian). Concourse runs at an abstraction layer above Docker, so providing any sort of magic there doesn't really make sense.

The one thing missing post-1.3 is that Docker requires you to set up cgroups yourself. I forgot how annoying that is. I wish they did what Guardian does and auto-configure it, but what can ya do.

So, the full set of instructions is:

Use or a build an image with docker in it, e.g. docker:dind. Run the following at the start of your task: https://github.com/concourse/docker-image-resource/blob/master/assets/common.sh#L1-L40 Spin up Docker with docker daemon &.

Then you can run docker-compose and friends as normal.

The downside of this is that you'll be fetching the images every time. #230 will address that.

In the long run, #324 (comment) is the direction I want to go.

See here https://github.com/concourse/concourse/issues/324

as in the accepted answer, the Slack archive data is deleted (due to Slack limit)

The docker image specialized for the usecase: https://github.com/meAmidos/dcind