stdout being buffered in docker container

Question

I'm not entirely sure what is going on here but it appears that stdout is being buffered when I run my code in a container, but not if I run it on the host or on OSX.

https://github.com/myles-mcdonnell/procwrap/blob/master/procwrap.go

The relevant piece (modified for brevity):

    cmd := exec.Command("ping", "127.0.0.1")

    logger := &lumberjack.Logger{
        Filename:   conf.LogFile,
        MaxSize:    conf.MaxLogSizeMb,
        MaxBackups: conf.MaxLogBackups,
        MaxAge:     conf.MaxLogAgeDays,
    }

    cmd.Stdout = io.MultiWriter(os.Stdout, logger)  

    err := cmd.Run()

Running in a container the sub process runs fine but I only see output (to stdout and the log file) at intervals, like a buffer is being flushed. Run it outside of a container and it outputs as it is generated. I'm using go 1.6.3. I see the same behaviour running the container interactively and in the background.

Docker version:

Client:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 21:49:11 2016
 OS/Arch:      darwin/amd64

Server:
 Version:      1.12.0
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   8eab29e
 Built:        Thu Jul 28 23:54:00 2016
 OS/Arch:      linux/amd64

======= UPDATE ======

I'm seeing different behaviour based on the base image. Running procwrap on a debian:wheezy base I get buffered output. Doing the same on ubuntu:trusty it's synchronous. Dockerfiles below, simply execute 'docker run {image_name}' on each to observe. Run procwrap on a wheezy VM (no docker involved) does not buffer the output.

TRUSTY:

FROM ubuntu:trusty
RUN apt-get update
RUN apt-get install -y curl
RUN apt-get install -y git
RUN curl -O https://storage.googleapis.com/golang/go1.6.3.linux-amd64.tar.gz
RUN tar -xvf go1.6.3.linux-amd64.tar.gz
RUN mv go /usr/local
ENV GOROOT=/usr/local/go
RUN mkdir -p /go/src/github.com/myles-mcdonnell
ENV GOPATH=/go
ENV PATH=$PATH:$GOPATH/bin:$GOROOT/bin
RUN go get github.com/myles-mcdonnell/procwrap
WORKDIR /go/src/github.com/myles-mcdonnell/procwrap
CMD procwrap -v

WHEEZY:

FROM debian:wheezy
RUN apt-get update
RUN apt-get install -y curl
RUN apt-get install -y git
RUN curl -O https://storage.googleapis.com/golang/go1.6.3.linux-amd64.tar.gz
RUN tar -xvf go1.6.3.linux-amd64.tar.gz
RUN mv go /usr/local
ENV GOROOT=/usr/local/go
RUN mkdir -p /go/src/github.com/myles-mcdonnell
ENV GOPATH=/go
ENV PATH=$PATH:$GOPATH/bin:$GOROOT/bin
RUN go get github.com/myles-mcdonnell/procwrap
WORKDIR /go/src/github.com/myles-mcdonnell/procwrap
CMD procwrap -v

Answer 1

I faced same issue and looks like root cause is "stdout buffering". You can start reading http://www.pixelbeat.org/programming/stdio_buffering/ and google in case of any additional question. I failed to resolve it by "stdbuf" or any linux/app settings and changed source code of the console app I want to dockerize to proceed with file flush on each write.

(my configuration is strange because of a lot of reason, but it is: the app has configured log output to the file named "/dev/stdout" and it performs file_flush on each log write and docker can show messages in real time on "docker logs -f XXXXX")

Answer 2

Is this what you're asking?

Why am I seeing my output in chunks? Is it being buffered and flushed somewhere?

Yes. Your Docker container is running on a Docker engine (possibly wrapped by a Docker machine) and your commands are being run from a client. Even if your engine is installed locally and you're running your container in interactive mode, the commands you're entering and the output you're seeing are still basically the results of HTTP requests against localhost. (Or Unix Socket or TTY communications.) (I'm simplifying.) Docker is routing I/O through your network drivers, i.e. its stdout has to be flushed to a (virtual?) network connection, and then the client flushes that to your stdout.

In background mode, the engine captures stdout and (by default) writes to a file. The docker logs command sends a request from your client to the engine, and the engine responds by reading back the contents of that file.