Tensorboard/tensorflow with s3 logdir - curl returned error code 6

Question

Been trying with numerous settings/env-vars/tf-versions but won't work..

On my local machine this works: AWS_ACCESS_KEY_ID=XXX AWS_SECRET_ACCESS_KEY=XXX AWS_REGION=eu-west-1 tensorboard --logdir="s3://my-bucket/tflogs/"

On a AWS instance this will throw:

I tensorflow/core/platform/s3/aws_logging.cc:54] Creating HttpClient with max connections2 and scheme http
I tensorflow/core/platform/s3/aws_logging.cc:54] Initializing CurlHandleContainer with size 2
I tensorflow/core/platform/s3/aws_logging.cc:54] Creating Instance with default EC2MetadataClient and refresh rate 900000
I tensorflow/core/platform/s3/aws_logging.cc:54] Found secret key
I tensorflow/core/platform/s3/aws_logging.cc:54] Initializing CurlHandleContainer with size 25
I tensorflow/core/platform/s3/aws_logging.cc:54] Found secret key
I tensorflow/core/platform/s3/aws_logging.cc:54] Pool grown by 2
I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.
E tensorflow/core/platform/s3/aws_logging.cc:60] Curl returned error code 6
W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
W tensorflow/core/platform/s3/aws_logging.cc:57] Request failed, now waiting 0 ms before attempting again.
I tensorflow/core/platform/s3/aws_logging.cc:54] Found secret key
2018-08-14 16:32:18.725199: I tensorflow/core/platform/s3/aws_logging.cc:54] Connection has been released. Continuing.
E tensorflow/core/platform/s3/aws_logging.cc:60] Curl returned error code 6
W tensorflow/core/platform/s3/aws_logging.cc:57] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.

Didn't find any hints in https://github.com/tensorflow/tensorflow/issues/16397

And no definitive clue where the diff could be, I have made sure to have the same tensorflow/tensorboard version (1.8.0). Also happens running tensorflow with s3 tensorboard logdir specified.

Answer 1

Solved it like this:

export AWS_ACCESS_KEY_ID=<access key id>
export AWS_SECRET_ACCESS_KEY=<secret access key>
export AWS_REGION=us-west-2
export S3_REGION=us-west-2
export S3_ENDPOINT=s3.us-west-2.amazonaws.com
export S3_USE_HTTPS=1
export S3_VERIFY_SSL=0

tensorboard --logdir=s3://<path>