Docker daemon (not containers) can't read environment variables

Question

Trying to configure a container running outside of GCP to log to Google Cloud Platform (StackDriver). One requirement is that the Docker daemon is able to locate the environment variable GOOGLE_APPLICATION_CREDENTIALS so it can authenticate. One would assume that the following would work, but it doesn't:

GOOGLE_APPLICATION_CREDENTIALS=/usr/local/keys/project-1.json docker run --log-driver=gcplogs ...

That outputs:

ERROR: for api Cannot start service api:
       failed to initialize logging driver: google: could not find default credentials.
       See https://developers.google.com/accounts/docs/application-default-credentials
       for more information.

Haven't found any documentation on how to set that directly on daemon.json, but I don't want that either because I might have different containers logging to different GCP projects.

I've tried this on Mac (docker desktop) and Debian.

Answer 1

This is question that keeps coming back. What is happening here is that environment variable GOOGLE_APPLICATION_CREDENTIALS is loaded by the system docker daemon. System daemons don't see the environment variables set in the user login. What you need to do is set the GOOGLE_APPLICATION_CREDENTIALS at the system level.

Here is how to do that in Ubuntu(Systemd):

$ sudo mkdir -p /etc/systemd/system/docker.service.d

Create /etc/systemd/system/docker.service.d/env.conf with the following content:

[Service]
Environment="GOOGLE_APPLICATION_CREDENTIALS=/path/to/file.json"  

Apply the changes.

$ sudo systemctl daemon-reload

Once done restart docker/containerd daemons

$ sudo systemctl restart containerd
$ sudo systemctl restart docker

Test the gcplogs driver

docker run --log-driver=gcplogs --log-opt gcp-project="my-project" hello-world