Setting up Docker with Knex.js and PostgreSQL

Question

Today I've been trying to setup Docker for my GraphQL API that runs with Knex.js, PostgreSQL and Node.js. The problem that I'm facing is that Knex.js is timing out when trying to access data in my database. I believe that it's due to my incorrect way of trying to link them together.

I've really tried to do this on my own, but I can't figure this out. I'd like to walk through each file that play a part of making this work so (hopefully) someone could spot my mistake(s) and help me.

knexfile.js

In my knexfile I have a connection key which used to look like this:

connection: 'postgres://localhost/devblog'

This worked just fine, but this wont work if I want to use Docker. So I modified it a bit and ended up with this:

connection: {
  host: 'db' || 'localhost',
  port: process.env.DB_PORT || 5432,
  user: process.env.DB_USER || 'postgres',
  password: process.env.DB_PASSWORD || undefined,
  database: process.env.DATABASE // DATABASE = devblog
}

I've noticed that something is wrong with host. Since it always times-out when I have something else (in this case, db) than localhost.

Dockerfile

My Dockerfile looks like this:

FROM node:9
WORKDIR /app
COPY package-lock.json /app
COPY package.json /app
RUN npm install
COPY dist /app
COPY wait-for-it.sh /app
CMD node index.js
EXPOSE 3010

docker-compose.yml

And this file looks like this:

version: "3"
services:
  redis:
    image: redis
    networks:
      - webnet
  db:
    image: postgres
    networks:
      - webnet
    environment:
      POSTGRES_PASSWORD: password
      POSTGRES_USER: martinnord
      POSTGRES_DB: devblog
  web:
    image: node:8-alpine
    command: "node /app/dist/index.js"
    volumes:
      - ".:/app"
    working_dir: "/app"
    depends_on:
      - "db"
    ports:
      - "3010:3010"
    environment:
      DB_PASSWORD: password
      DB_USER: martinnord
      DB_NAME: devblog
      DB_HOST: db
      REDIS_HOST: redis
networks:
  webnet:

When I try to run this with docker-compose up I get the following output:

Starting backend_db_1 ... done
Starting backend_web_1 ... done
Attaching to backend_db_1, backend_redis_1, backend_web_1
redis_1  | 1:C 12 Feb 16:05:21.303 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1  | 1:C 12 Feb 16:05:21.303 # Redis version=4.0.8, bits=64, commit=00000000, modified=0, pid=1, just started
db_1     | 2018-02-12 16:05:21.337 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
redis_1  | 1:C 12 Feb 16:05:21.303 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1  | 1:M 12 Feb 16:05:21.311 * Running mode=standalone, port=6379.
db_1     | 2018-02-12 16:05:21.338 UTC [1] LOG:  listening on IPv6 address "::", port 5432
redis_1  | 1:M 12 Feb 16:05:21.311 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1  | 1:M 12 Feb 16:05:21.314 # Server initialized
redis_1  | 1:M 12 Feb 16:05:21.315 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
db_1     | 2018-02-12 16:05:21.348 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1     | 2018-02-12 16:05:21.367 UTC [20] LOG:  database system was shut down at 2018-02-12 16:01:17 UTC
redis_1  | 1:M 12 Feb 16:05:21.317 * DB loaded from disk: 0.002 seconds
redis_1  | 1:M 12 Feb 16:05:21.317 * Ready to accept connections
db_1     | 2018-02-12 16:05:21.374 UTC [1] LOG:  database system is ready to accept connections
web_1    | DB_HOST db
web_1    |
web_1    |       App listening on 3010
web_1    |       Env: undefined

But when I try to make a query with GraphQL I get: "message": "Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?"

I really don't know why this is not working for me and it's driving me nuts. If anyone could help me with this I would be delighted. I have also added a link to my project below.

Thanks a lot for reading! Cheers.

LINK TO PROJECT: https://github.com/Martinnord/DevBlog/tree/master/backend

Updated docker-compose file:

version: "3"
services:
  redis:
    image: redis
    networks:
      - webnet
  db:
    image: postgres
    networks:
      - webnet
    environment:
      POSTGRES_PASSWORD: postgres
      POSTGRES_USER: martinnord
      POSTGRES_DB: devblog
    ports:
      - "15432:5432"
  web:
    image: devblog-server
    ports:
      - "3010:3010"
    networks:
      - webnet
    environment:
      DB_PASSWORD: password
      DB_USER: martinnord
      DB_NAME: devblog
      DB_HOST: db
      REDIS_HOST: redis
    command: ["./wait-for-it.sh", "db:5432", "--", "node", "index.js"]
networks:
    webnet:

Answer 1

Maybe like this:

version: "3"
services:
  redis:
    image: redis
  db:
    image: postgres
    environment:
      POSTGRES_PASSWORD: password
      POSTGRES_USER: martinnord
      POSTGRES_DB: devblog
    ports:
      - "15432:5432"
  web:
    image: node:8-alpine
    command: "node /app/dist/index.js"
    volumes:
      - ".:/app"
    working_dir: "/app"
    depends_on:
      - "db"
    ports:
      - "3010:3010"
    links:
      - "db"
      - "redis"
    environment:
      DB_PASSWORD: password
      DB_USER: martinnord
      DB_NAME: devblog
      DB_HOST: db
      REDIS_HOST: redis

And you should be able to connect from webapp to postgres with: postgres://martinnord:password@db/devblog

or

connection: {
  host: process.DB_HOST,
  port: process.env.DB_PORT || 5432,
  user: process.env.DB_USER || 'postgres',
  password: process.env.DB_PASSWORD,
  database: process.env.DB_NAME || 'postgres'
}

I also added row that exposes postgres running in docker to a port 15432 so you can try to connect it first directly from your host machine with

psql postgres://martinnord:password@localhost:15432/devblog