How to detect if docker container crashed

Question

I am running a process every 5 minutes and checking to make sure every container. If the container does not respond I can flag it as down. I have the containers IP address and I loop through each ip and check if it responds to a ping. If not I flag it as down. Is there a better way to do this? My code:

@Transactional
@Scheduled(fixedRate = 1000 * 60)   //5 min
public void monitorHosts(){
    Iterable<Ncl> ncls = nclRepository.findAll();

    for(Ncl ncl: ncls){
        for(String host: ncl.getHosts()){
            Boolean isHostAlive = isHostAlive(host);
            if(!isHostAlive){
                Ncl nclWorking = nclRepository.findOne(ncl.getUuid());
                if(nclWorking != null){
                    Set<String> hosts = nclWorking.getHosts().stream().filter(x -> x.equals(host)).collect(Collectors.toSet());
                    nclWorking.getHosts().clear();
                    nclWorking = nclRepository.save(nclWorking);
                    nclWorking.setHosts(hosts);
                    nclRepository.save(nclWorking);
                }
            }
        }
    }
}

private Boolean isHostAlive(String host){
    try{
        InetAddress address = InetAddress.getByName(host);
        boolean reachable = address.isReachable(10000);
        return reachable;
    } catch (Exception e){
        e.printStackTrace();
        return false;
    }
}

Answer 1

It mostly depends on what you need to do with the information about your containers.

There is a number of monitoring solutions available, which can monitor your containers and notify some one if there are some troubles.

If you have to use this info in some application, then you can use some solutions like Consul.io and let them check your services statuses, not containers (in most cases man aware of the service availability in the container, not container itself). Or you can use docker-api for Java, because ICMP-protocol is not always a good solution, especially in distributed networks.

Answer 2

I would use docker events

extract from the doc

https://docs.docker.com/engine/reference/commandline/events/#examples

use something like

docker events --filter 'event=stop'