Database migrations in docker swarm mode

Question

I have an application that consists of simple Node app and Mongo db. I wonder, how could I run database migrations in docker swarm mode?

Without swarm mode I run migrations by stopping first the old version of application, running one-off migration command with new version of application and then finally starting a new version of app:

# Setup is roughly the following
$ docker network create appnet
$ docker run -d --name db --net appnet db:1
$ docker run -d --name app --net appnet -p 80:80 app:1

# Update process
$ docker stop app && docker rm app
$ docker run --rm --net appnet app:2 npm run migrate
$ docker run -d --name app --net appnet -p 80:80 app:2

Now I'm testing the setup in docker swarm mode so that I could easily scale app. The problem is that in swarm mode one can't start containers in swarm network and thus I can't reach the db to run migrations:

$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
6jtmtihmrcjl        appnet              overlay             swarm

# Trying to replicate the manual migration process in swarm mode
$ docker service scale app=0
$ docker run --rm --net appnet app:2 npm run migrate
docker: Error response from daemon: swarm-scoped network (appnet) is not compatible with `docker create` or `docker run`. This network can only be used by a docker service.

I don't want to run the migration command during app startup either, as there might be several instances launching and that would potentially screw the database. Automatic migrations are scary, so I want to avoid them at all costs.

Do you have any idea how to implement manual migration step in docker swarm mode?

Edit

I found out a dirty hack that allows to replicate the original workflow. Idea is to create a new service with custom command and remove it when one of its tasks is finished. This is far from pleasant usage, better alternatives are more than welcome!

$ docker service scale app=0
$ docker service create --name app-migrator --network appnet app:2 npm run migrate

# Check when the first app-migrator task is finished and check its output
$ docker service ps app-migrator
$ docker logs <container id from app-migrator>
$ docker service rm app-migrator

# Ready to update the app
$ docker service update --image app:2 --replicas 2 app

Answer 1

I believe you can fix this problem by making your overlay network, appnet, attachable. This can be accomplished with the following command:

docker network create --driver overlay --attachable appnet

This should fix the swarm-scoped network error and and allow you to run migrations