Menu
Docker daemon is unable to parse the json log and throws an unexpected EOF error. We are investigating the root cause of the issue.
Environment:
Docker daemon log:
Apr 29 14:31:05 Prod-IS dockerd[30810]: time="2020-04-29T14:31:05.070677515+02:00" level=warning msg="got error while decoding json" error="unexpected EOF" retries=345
Apr 29 14:31:05 Prod-IS dockerd[30810]: time="2020-04-29T14:31:05.070695689+02:00" level=warning msg="got error while decoding json" error="unexpected EOF" retries=346
Apr 29 14:31:05 Prod-IS dockerd[30810]: time="2020-04-29T14:31:05.070712630+02:00" level=warning msg="got error while decoding json" error="unexpected EOF" retries=347
Apr 29 14:31:05 Prod-IS dockerd[30810]: time="2020-04-29T14:31:05.070732299+02:00" level=warning msg="got error while decoding json" error="unexpected EOF" retries=348
Apr 29 14:31:05 Prod-IS dockerd[30810]: time="2020-04-29T14:31:05.070755016+02:00" level=warning msg="got error while decoding json" error="unexpected EOF" retries=349
Apr 29 14:31:05 Prod-IS dockerd[30810]: time="2020-04-29T14:31:05.070773699+02:00" level=warning msg="got error while decoding json" error="unexpected EOF" retries=350
Docker daemon configuration
"log-driver": "json-file",
"log-opts": {
"mode": "non-blocking",
"max-size": "500m",
"max-file": "3"
}
Storage space is available on partition where docker writes its logs.
Your insight in the matter is most welcome.
379
The Docker daemon log can be viewed by using one of the following methods: By running journalctl -u docker.service on Linux systems using systemctl. /var/log/messages , /var/log/daemon.log , or /var/log/docker.log on older Linux systems.
Conclusion. The Docker daemon is a backend service of Docker that controls the Docker container. To resolve the Docker daemon is not running error, you first need to verify if the service of Docker Desktop is running or not. If the service is running then update the WSL package.
Docker container logs are generated by the Docker containers. They need to be collected directly from the containers. Any messages that a container sends to stdout or stderr is logged then passed on to a logging driver that forwards them to a remote destination of your choosing.
Issue=Docker Engine Error json-file: fix sporadic unexpected EOF errors
I had spent some time researching and trying to find errors in the docker logs.
There is a bug in docker engine 20.10.5 which is the version running in docker. A Github PR fixes this. https://github.com/moby/moby/pull/42104
I found the error while running
$ journalctl -u docker.service
level=warning msg="got error while decoding json" error="unexpected EOF" retries=19999
Docker driver for logs has a bug and it wont handle some log formatting. The bug then floods the system's log with retries messages. This situation generates a very annoying CPU load.
Docker engine latest is 20.10.7. The issue has been fixed in release Docker engine 20.10.6 https://docs.docker.com/engine/release-notes/#20106
74
The aforementioned issue will cause an abnormal spike in CPU usage of the docker (version <= 19) process and constantly stay at that level until you fix the logs.
To fix the logs & cpu usage, you've to truncate all the container logs (since we are unable to determine which container has faulty logs). It works only if your logs are in json format:
truncate -s 0 /var/lib/docker/containers/*/*-json.log
Please see Andre's answer for a permanent fix.
36
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With