redirect log files created in a docker container to stdout / stderr

Question

I have the following Dockerfile:

FROM ubuntu:16.04
RUN apt-get update
VOLUME ["/LOGS"]
COPY ./testServer .
ENTRYPOINT ./testServer 8600

"testServer" has log files that are being written to. They are located in the directory "LOGS". Each time "testServer" is started, a new log is created. What I wanted to do was to "tail" the latest log file in the directory to stdout / stderr.

I tried adding:

CMD ["/bin/sh", "-c", "tail $( ls -Art /LOGS | tail -n 1 ) > out_server.log 2>&1"]

to the Dockerfile (and rebuilt the image thereafter) but it did not work.

How can this be done?

TIA

Answer 1

Instead you using tail, you can symlink the log file to the container process's stdout. To do so, you need to wrap your executable in a separate script so it gets launched as a process separate from the container's main process.

Script to execute:

#!/bin/bash

# The application log will be redirected to the main docker container process's stdout, so # that it will show up in the container logs
touch /my/app/application.log
ln -sf /proc/1/fd/1 /my/app/application.log

# Execute my app
./testServer 8600

And in the docker file just copy and execute the script

COPY start_server.sh /the/above/script/start_server.sh
CMD ["/bin/bash", "/the/above/script/start_server.sh"]

Answer 2

There are two issues here.

1. You have an ENTRYPOINT defined and you are trying to run a command with CMD. Docker starts a container with a single process, and when you define both, CMD is appended to ENTRYPOINT as additional cli args. What the container is running as pid 1 is:

   /bin/sh -c './testServer 8600 /bin/sh -c "tail $( ls -Art /LOGS | tail -n 1 ) > out_server.log 2>&1"'

   Unless testServer runs the extra args, they'll never get used.

2. If the command you were running did work, it would output everything to /out_server.log inside the container, not to stdout, and it would stop as soon as it reached the end of the input. If that's your pid 1, the container would also exit at that point.

To fix this, you can create an entrypoint.sh to be something like:

#!/bin/sh

./testServer 8600 &
sleep 2 # give testServer time to create the newest log
exec tail -f $( ls -Art /LOGS | tail -n 1 )

That entrypoint starts up testServer in the background and then runs the tail with an exec. The exec replaces pid 1 so that signals are passed through.

Update your Dockerfile to:

FROM ubuntu:16.04

# This apt-get line shouldn't be needed unless something else 
# needs the possibly outdated package repo list
# RUN apt-get update

# Removing this volume, you can create it from a docker-compose.yml
# VOLUME ["/LOGS"]

COPY entrypoint.sh testServer /
RUN chmod 755 /entrypoint.sh /testServer
ENTRYPOINT [ "/entrypoint.sh" ]

For more details on why I removed the VOLUME line, see my blog post here.