AWS Elasticsearch VPC connectivity

Question

I've created an Elasticsearch domain in AWS.

It's added to my VPC inside a public subnet and I've attached a security group which is currently completely open.

I have this policy attached also:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:eu-central-1:ACCOUNT_ID:domain/DOMAIN_NAME/*"
    }
  ]
}

I am trying to access an endpoint locally but it doesn't seem to be allowed.

The Kibana URL for example is:

https://vpc-bla.bla.bla.eu-central-1.es.amazonaws.com/_plugin/kibana/

Any idea why I'm not able to access this URL?

Answer 1

After much trial and error, I found the URL generated by ES is internal and cannot be opened to the internet easily via security groups.

Instead, I deployed an simple nginx proxy which forwarded public DNS requests eg es.mydns.com to the internal DNS eg vpc....eu-central-1.es.amazonaws.com/_plugin/kibana/

More nginx info here.

Answer 2

VPC Endpoint cannot be accessed outside the subnets that you associated with the elastic-search domain.

you can try performing curl from any EC2 instances that is part of the same subnet that you associated with elastic-search, it should work.

If you need to access the endpoint from internet, then don't create VPC endpoint elastic-search instead create elastic-search domain with internet access. You can specify whether you want VPC or internet accessible cluster when creating the ES domain.