Kibana is not connecting to elasticsearch docker

Question

New to ELK stack + docker.

Trying to setup ELK setup locally in docker.

The command used is

docker network create elasticnew --driver=bridge
docker run --network=elasticnew --name elasticsearchnode -p 9200:9200 -p 9300:9300 docker.elastic.co/elasticsearch/elasticsearch:6.6.1
docker run --network=elasticnew --name kibana -e ELASTICSEARCH_URL=http://elasticsearchnode:9200 -p 5601:5601 docker.elastic.co/kibana/kibana:6.6.1

ElasticSeach is up and http://localhost:9200/ is giving json response.

But the kibana url (http://localhost:5601/) says "Kibana server is not ready yet" and the request keeps on spinning in browser.

While querying for docker container, it says,

CONTAINER ID:76fe82529fa9       
IMAGE:docker.elastic.co/kibana/kibana:6.6.1                                                      
CREATED:16 minutes ago
STATUS:Up 15 minutes 
PORTS:0.0.0.0:5601->5601/tcp                                            
NAMES:kibana

Edit on findings:-

1.Initially the command

docker logs kibana 

gives empty response immediately. Later on issuing this command, taking time exponentially to reply (Did not see the logs after 5 mins too.)

2.While localhost:9200 in browser too, replicates the above behaviour. (It seems something like, kibana is making the elastic search busy)

3.Did inspect on kibana container, and it also says the status as "running". Hereby shared logs too.

C:\Windows\system32>docker inspect kibana
[
{
    "Id": "da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52",
    "Created": "2019-02-25T18:39:43.1796454Z",
    "Path": "/usr/local/bin/kibana-docker",
    "Args": [],
    "State": {
        "Status": "running",
        "Running": true,
        "Paused": false,
        "Restarting": false,
        "OOMKilled": false,
        "Dead": false,
        "Pid": 3398,
        "ExitCode": 0,
        "Error": "",
        "StartedAt": "2019-02-25T18:39:48.7600134Z",
        "FinishedAt": "0001-01-01T00:00:00Z"
    },
    "Image": "sha256:b94222148a00695eb94bcf9fe5cce02547ffd963531709de15187bf8ade13ea0",
    "ResolvConfPath": "/var/lib/docker/containers/da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52/resolv.conf",
    "HostnamePath": "/var/lib/docker/containers/da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52/hostname",
    "HostsPath": "/var/lib/docker/containers/da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52/hosts",
    "LogPath": "/var/lib/docker/containers/da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52/da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52-json.log",
    "Name": "/kibana",
    "RestartCount": 0,
    "Driver": "overlay2",
    "Platform": "linux",
    "MountLabel": "",
    "ProcessLabel": "",
    "AppArmorProfile": "",
    "ExecIDs": null,
    "HostConfig": {
        "Binds": null,
        "ContainerIDFile": "",
        "LogConfig": {
            "Type": "json-file",
            "Config": {}
        },
        "NetworkMode": "elasticnew",
        "PortBindings": {
            "5601/tcp": [
                {
                    "HostIp": "",
                    "HostPort": "5601"
                }
            ]
        },
        "RestartPolicy": {
            "Name": "no",
            "MaximumRetryCount": 0
        },
        "AutoRemove": false,
        "VolumeDriver": "",
        "VolumesFrom": null,
        "CapAdd": null,
        "CapDrop": null,
        "Dns": [],
        "DnsOptions": [],
        "DnsSearch": [],
        "ExtraHosts": null,
        "GroupAdd": null,
        "IpcMode": "shareable",
        "Cgroup": "",
        "Links": null,
        "OomScoreAdj": 0,
        "PidMode": "",
        "Privileged": false,
        "PublishAllPorts": false,
        "ReadonlyRootfs": false,
        "SecurityOpt": null,
        "UTSMode": "",
        "UsernsMode": "",
        "ShmSize": 67108864,
        "Runtime": "runc",
        "ConsoleSize": [
            30,
            120
        ],
        "Isolation": "",
        "CpuShares": 0,
        "Memory": 0,
        "NanoCpus": 0,
        "CgroupParent": "",
        "BlkioWeight": 0,
        "BlkioWeightDevice": [],
        "BlkioDeviceReadBps": null,
        "BlkioDeviceWriteBps": null,
        "BlkioDeviceReadIOps": null,
        "BlkioDeviceWriteIOps": null,
        "CpuPeriod": 0,
        "CpuQuota": 0,
        "CpuRealtimePeriod": 0,
        "CpuRealtimeRuntime": 0,
        "CpusetCpus": "",
        "CpusetMems": "",
        "Devices": [],
        "DeviceCgroupRules": null,
        "DiskQuota": 0,
        "KernelMemory": 0,
        "MemoryReservation": 0,
        "MemorySwap": 0,
        "MemorySwappiness": null,
        "OomKillDisable": false,
        "PidsLimit": 0,
        "Ulimits": null,
        "CpuCount": 0,
        "CpuPercent": 0,
        "IOMaximumIOps": 0,
        "IOMaximumBandwidth": 0,
        "MaskedPaths": [
            "/proc/asound",
            "/proc/acpi",
            "/proc/kcore",
            "/proc/keys",
            "/proc/latency_stats",
            "/proc/timer_list",
            "/proc/timer_stats",
            "/proc/sched_debug",
            "/proc/scsi",
            "/sys/firmware"
        ],
        "ReadonlyPaths": [
            "/proc/bus",
            "/proc/fs",
            "/proc/irq",
            "/proc/sys",
            "/proc/sysrq-trigger"
        ]
    },
    "GraphDriver": {
        "Data": {
            "LowerDir": "/var/lib/docker/overlay2/156be4deb6525acdb0b6b32d757c5da721271aa61010fac1c3bdc89e8793d63d-init/diff:/var/lib/docker/overlay2/7afef12f62d8f016bd357422310f4cf3cf58f5b66b4bc03294684e970682a71f/diff:/var/lib/docker/overlay2/e86e9fc46fd2c18605165003912fd99161c6826dea9650fe3f7c591a06a13529/diff:/var/lib/docker/overlay2/a7e09d4df14208f4b48adf12b155a4ba0c1fb2ce5f66e2f4c2e2f2c6f030b9e2/diff:/var/lib/docker/overlay2/aaf3b5122fcff237d0f90bfe3d6ba778c4ff036f6d00db5aa6d301083038026d/diff:/var/lib/docker/overlay2/497685f1ed275b7ad5b6ca23beb3840542abb8e5ba38ac51d9b8fb20ec007c4b/diff:/var/lib/docker/overlay2/532a837d610aaeb5c0e5b98a2dc4df1d899a46559b8e4429ff14a25b5ba60c9a/diff:/var/lib/docker/overlay2/b94ac41fbb4f0941a19161f1d3289d218de10fc36c2037d0667249caf65e049b/diff:/var/lib/docker/overlay2/c9fe17bf9817837aec687bf5e14eeb1e9997ef92c4873c8cd9355d57a75aa71c/diff:/var/lib/docker/overlay2/6e3558bcf051abf5cf3066775f43e81d565158e3b2a9cb817a501b47bc7679a5/diff",
            "MergedDir": "/var/lib/docker/overlay2/156be4deb6525acdb0b6b32d757c5da721271aa61010fac1c3bdc89e8793d63d/merged",
            "UpperDir": "/var/lib/docker/overlay2/156be4deb6525acdb0b6b32d757c5da721271aa61010fac1c3bdc89e8793d63d/diff",
            "WorkDir": "/var/lib/docker/overlay2/156be4deb6525acdb0b6b32d757c5da721271aa61010fac1c3bdc89e8793d63d/work"
        },
        "Name": "overlay2"
    },
    "Mounts": [],
    "Config": {
        "Hostname": "da312a8bfbe5",
        "Domainname": "",
        "User": "1000",
        "AttachStdin": false,
        "AttachStdout": true,
        "AttachStderr": true,
        "ExposedPorts": {
            "5601/tcp": {}
        },
        "Tty": false,
        "OpenStdin": false,
        "StdinOnce": false,
        "Env": [
            "ELASTICSEARCH_URL=http://elasticsearchnode:9200",
            "PATH=/usr/share/kibana/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "ELASTIC_CONTAINER=true"
        ],
        "Cmd": [
            "/usr/local/bin/kibana-docker"
        ],
        "ArgsEscaped": true,
        "Image": "docker.elastic.co/kibana/kibana:6.6.1",
        "Volumes": null,
        "WorkingDir": "/usr/share/kibana",
        "Entrypoint": null,
        "OnBuild": null,
        "Labels": {
            "license": "Elastic License",
            "org.label-schema.build-date": "20181205",
            "org.label-schema.license": "GPLv2",
            "org.label-schema.name": "kibana",
            "org.label-schema.schema-version": "1.0",
            "org.label-schema.url": "https://www.elastic.co/products/kibana",
            "org.label-schema.vcs-url": "https://github.com/elastic/kibana-docker",
            "org.label-schema.vendor": "Elastic",
            "org.label-schema.version": "6.6.1"
        }
    },
    "NetworkSettings": {
        "Bridge": "",
        "SandboxID": "3bb18d0b036874dabdc526080d6cf25ac3e53147b7b2fd44ce77ce58b6104900",
        "HairpinMode": false,
        "LinkLocalIPv6Address": "",
        "LinkLocalIPv6PrefixLen": 0,
        "Ports": {
            "5601/tcp": [
                {
                    "HostIp": "0.0.0.0",
                    "HostPort": "5601"
                }
            ]
        },
        "SandboxKey": "/var/run/docker/netns/3bb18d0b0368",
        "SecondaryIPAddresses": null,
        "SecondaryIPv6Addresses": null,
        "EndpointID": "",
        "Gateway": "",
        "GlobalIPv6Address": "",
        "GlobalIPv6PrefixLen": 0,
        "IPAddress": "",
        "IPPrefixLen": 0,
        "IPv6Gateway": "",
        "MacAddress": "",
        "Networks": {
            "elasticnew": {
                "IPAMConfig": null,
                "Links": null,
                "Aliases": [
                    "da312a8bfbe5"
                ],
                "NetworkID": "d6e0ffb7617a1a2dd4f1a15c10f7255ed560c980ad6ccf1fb573a292d4515a9c",
                "EndpointID": "91359e830287666cf834372691e2761ea239450c832551360af5b7870546a869",
                "Gateway": "172.19.0.1",
                "IPAddress": "172.19.0.3",
                "IPPrefixLen": 16,
                "IPv6Gateway": "",
                "GlobalIPv6Address": "",
                "GlobalIPv6PrefixLen": 0,
                "MacAddress": "02:42:ac:13:00:03",
                "DriverOpts": null
            }
        }
    }
}
]

Regret for those big logs.

PS: I am using Win 10 Enterprise, 6GB of RAM and after running the kibana and elastic search docker containers, the RAM usage was close to 70% and above.

Could some one share some input here on

1. Why kibana fails to connect with elastic search?
2. Why elastic search too fails to respond after kibana fails to run?

Answer 1

In my case the problem was due to the naming of my container... Notice --name elasticsearch here:

docker run -d --name elasticsearch --net elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.4.0

docker run -d --name kibana --net elasticsearch -p 5601:5601 kibana:7.4.0

I was giving the container another name and kibana was complaining that it was not able to connect. Changed it to elasticsearch then waited like 1 minute for the time for kibana to create its indexes ad install plugins.

Just check kibanas' logs until it says "Server running at http://0:5601"

docker logs kibana