Output of `tail -f` at the end of a docker CMD is not showing

Question

Using Docker for Mac 1.13.1 with the following Dockerfile:

FROM ubuntu:latest
MAINTAINER [email protected]

#Install packages and clean downloaded packages in the lowest layer
RUN apt-get update && apt-get -y install cron && rm -rf /var/lib/apt/lists/*

# Add crontab file in the cron directory
ADD crontab /etc/cron.d/hello-cron

# Give execution rights on the cron job and create the log file to tail in the next layer
RUN chmod 0644 /etc/cron.d/hello-cron && touch /var/log/cron.log

# Run the command on container startup
CMD echo "starting" && echo "continuing" && (cron) && echo "tailing..."  && tail -f /var/log/cron.log

With a contab file of:

* * * * * root echo "Hello world `date`" >> /var/log/cron.log 2>&1
# Don't remove the empty line at the end of this file. It is required to run the cron job

When I build and run it with:

docker build -t docker-cron-master .
docker run docker-cron-master

I see the output:

docker run docker-cron-master
starting
continuing
tailing...

If I wait a minute the tail -f output doesn't appear. Yet if I login to the running container and tail the file I can see the contents:

$ docker exec -it 4eda6b1fc6ce bash
root@4eda6b1fc6ce:/# tail -f /var/log/cron.log
Hello world Fri May  5 09:53:01 UTC 2017
Hello world Fri May  5 09:54:01 UTC 2017

I tried adding another echo at the end of the CMD to see if it was only the last command who's STDOUT was being swallowed but that didn't help.

I have posted the code is on github at https://github.com/simbo1905/docker-cron

Thanks!

Answer 1

The docker filesystem uses copy-on-write with it's layered union fs. So when you write to a file that's part of the image, it will first make a copy of that file to the container filesystem which is a layer above all the image layers.

What that means is when you append a line to the /var/log/cron.log, it will get a new inode in the filesystem and the file that the tail command is following at is no longer the one you see when you docker exec into the container. You can solve that with a minor change to append "nothing" to the file which also modifies the last update timestamp which forces a copy-on-write:

CMD echo "starting" && echo "continuing" && (cron) \
 && echo "tailing..." && : >> /var/log/cron.log && tail -f /var/log/cron.log

I put together a gist that goes through this issue with a lot more detail over here: https://gist.github.com/sudo-bmitch/f91a943174d6aff5a57904485670a9eb