Postgres Docker: "postgres: could not access the server configuration file "/var/lib/postgresql/data/postgresql.conf": No such file or directory"

Question

I am having weird issues with official postgres docker image. Most of the time it works fine, if I shut down the container and launch it again, I sometimes get this error but it's not every time:

PostgreSQL Database directory appears to contain a database; Skipping initialization

postgres: could not access the server configuration file "/var/lib/postgresql/data/postgresql.conf": No such file or directory

I am launching postgres image using this command:

 export $(grep -v '^#' .env | xargs) && docker run --rm --name postgres \
  -e POSTGRES_USER=$POSTGRES_USER \
  -e POSTGRES_DB=$POSTGRES_DB \
  -e POSTGRES_PASSWORD=$POSTGRES_PASSWORD \
  -p $POSTGRES_PORT:$POSTGRES_PORT \
  -v $POSTGRES_DEVELOPMENT_DATA:/var/lib/postgresql/data \
  postgres

I keep variables in .env file, they look like this:

POSTGRES_USER=custom-db
POSTGRES_DB=custom-db
POSTGRES_PASSWORD=12345678
POSTGRES_PORT=5432
POSTGRES_DEVELOPMENT_DATA=/tmp/custom-db-pgdata

When I try to echo variables the values are there so I don't think I'm passing null values to docker env variables.

The directory on my host machine looks something like this:

/tmp/custom-db-pgdata
├── base
│   ├── 1
│   ├── 13407
│   ├── 13408
│   └── 16384
├── global
├── pg_logical
├── pg_multixact
│   ├── members
│   └── offsets
├── pg_notify
├── pg_stat
├── pg_stat_tmp
├── pg_subtrans
├── pg_wal
│   └── archive_status
└── pg_xact

Answer 1

If it's inconsistent in how it works between executions on the same machine and same session (aka without rebooting) then something isn't mapping your directories properly. Finding what it is that's breaking will be difficult, more so since you're on a Mac. Docker on a Mac you has the extra bonus of running through a VM, so docker is mapping your local drive/path through to the VM and then mapping that into the container image, so there are two different layers where things can go wrong.

Dario has the right idea in his clarifying comments, you shouldn't rely on /tmp since that also has Mac Magic to it. It's actually /var/private/somegarbagestring and is different every bootup. Try switching to a /Users/$USER/dbpath folder and move your data to that, so at least you're debugging with one less layer of magic between data and database.