How to view docker-compose healthcheck logs?

Question

Inside my docker-compose.yml, I have the following service healthcheck section. I want to know if MariaDB is actually ready to handle queries. A service named cmd is configured to depend on condition: service_healthy.

  db:     image: mariadb:10     environment:       MYSQL_RANDOM_ROOT_PASSWORD: 1       MYSQL_USER: user       MYSQL_PASSWORD: password       MYSQL_DATABASE: database     healthcheck:       test: ["CMD", "mysql", "--user=user", "--password=password", "--execute='SELECT 1'", "--host=127.0.0.1", "--port=3306"]       interval: 1s       retries: 30 

This healthcheck does not work, shows that the service is unhealthy.

How do I check the output of the test CMD?

Answer 1

docker-compose ps will indicate the status of each service, including its health if healthcheck is defined. It's good for a basic overview.

% docker-compose ps                 Name                                Command                       State                                       Ports                             ---------------------------------------------------------------------------------------------------------------------------------------------------------------- remix-theme-editor_analytics_1            /bin/sh -c /analytics/run. ...   Up                                                                                    remix-theme-editor_base_1                 /bin/bash                        Exit 0                                                                                remix-theme-editor_flower_1               /entrypoint --environment  ...   Exit 137                                                                              remix-theme-editor_frontend_1             /bin/sh -c perl -p -i -e ' ...   Exit 137                                                                              remix-theme-editor_js-app_1               npm run                          Exit 0                                                                                remix-theme-editor_mq_1                   docker-entrypoint.sh rabbi ...   Up (healthy)            15671/tcp, 15672/tcp, 25672/tcp, 4369/tcp, 5671/tcp, 5672/tcp remix-theme-editor_mysql-migration_1      /entrypoint_mysql-migratio ...   Exit 0                                                                                remix-theme-editor_mysql_1                /bin/sh -c /entrypoint_wra ...   Up (health: starting)   127.0.0.2:3308->3306/tcp                                      remix-theme-editor_page-renderer_1        npm run start:watch              Up                                                                                    remix-theme-editor_python-app_1           /entrypoint                      Exit 2                                                                                remix-theme-editor_redis_1                docker-entrypoint.sh /bin/ ...   Up (health: starting)   6379/tcp                                                      remix-theme-editor_scheduler_1            /entrypoint --environment  ...   Exit 137                                                                              remix-theme-editor_socket_1               /entrypoint --environment  ...   Exit 1                                                                                remix-theme-editor_static-builder_1       npm run watch                    Up                                                                                    remix-theme-editor_static-http_1          nginx -g daemon off;             Up                      127.0.0.2:6544->443/tcp, 80/tcp                               remix-theme-editor_web_1                  /entrypoint --environment  ...   Exit 1                                                                                remix-theme-editor_worker_1               /entrypoint --environment  ...   Exit 1                                                                                remix-theme-editor_worker_screenshots_1   /entrypoint --environment  ...   Exit 1      

If you want more details, use docker inspect in conjuction with docker ps -q <service-name>.

% docker inspect --format "{{json .State.Health }}" $(docker-compose ps -q mq) | jq {   "Status": "starting",   "FailingStreak": 48,   "Log": [     {       "Start": "2018-10-03T00:40:18.671527745-05:00",       "End": "2018-10-03T00:40:18.71729051-05:00",       "ExitCode": -1,       "Output": "OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused \"exec: \\\"nc\\\": executable file not found in $PATH\": unknown"     }, ... 

You can always debug the healthcheck yourself by simply executing your healthcheck code yourself. For example:

% docker exec -it $(docker-compose ps -q socket) nc -w2 127.0.0.1 5672 (UNKNOWN) [127.0.0.1] 5672 (?) : Connection refused 

You can also do the same in shell:

% docker exec -it $(docker-compose ps -q socket) bash root@b5da5207d344:~/src# nc -w2 127.0.0.1 5672 (UNKNOWN) [127.0.0.1] 5672 (?) : Connection refused root@b5da5207d344:~/src# echo $? 1 

Finally, you can simply use docker-compose up in the first terminal window, and docker-compose logs -f in another. This will display all logs from docker-compose-managed containers.

Answer 2

You can use:

docker inspect --format "{{json .State.Health }}" <container name> | jq 

Output:

{     "Status": "unhealthy",     "FailingStreak": 63,     "Log": [         {             "Start": "2017-03-11T20:49:19.668895201+03:30",             "End": "2017-03-11T20:49:19.735722044+03:30",             "ExitCode": 1,             "Output": "ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''SELECT 1'' at line 1\n"         }     ] } 

And look for the output section.

To get the Output only:

docker inspect --format "{{json .State.Health }}" mariadb_db_1 | jq '.Log[].Output' 

For swarm mode use the folling format (thanks for pointing it out @shotgunner):

{{json.Spec.TaskTemplate.ContainerSpec.Healthcheck}}

Feel free to swap jq for whatever tool you use for json pretty print.