Connecting to scality/s3 server between docker containers

Question

We are using a python based solution which shall load and store files from S3. For developing and local testing we are using a vagrant environment with docker and docker-compose. We have two docker-compose defintions - one for the assisting backend services (mongo, restheart, redis and s3) and the other one containing the python based REST API exposing solution using the backend services.

When our "front-end" docker-compose group interacts with restheart this works fine (using the name of the restheart container as server host in http calls). When we are doing the same with scality/s3 server this does not work.

The interesting part is, that we have created a test suite for using the scality/s3 server from a python test suite running on the host (windows10) over the forwarded ports through vagrant to the docker container of scality/s3 server within the docker-compose group. We used the endpoint_url localhost and it works perfect.

In the error case (when frontend web service wants to write to S3) the "frontend" service always responds with: botocore.exceptions.ClientError: An error occurred (InvalidURI) when calling the CreateBucket operation: Could not parse the specified URI. Check your restEndpoints configuration. And the s3server always responds with http 400 and the message:

s3server | {"name":"S3","clientIP":"::ffff:172.20.0.7","clientPort":49404,"httpMethod":"PUT","httpURL":"/raw-data","time":1521306054614,"req_id":"e385aae3c04d99fc824d","level":"info","message":"received request","hostname":"cdc8a2f93d2f","pid":83} s3server | {"name":"S3","bytesSent":233,"clientIP":"::ffff:172.20.0.7","clientPort":49404,"httpMethod":"PUT","httpURL":"/raw-data","httpCode":400,"time":1521306054639,"req_id":"e385aae3c04d99fc824d","elapsed_ms":25.907569,"level":"info","message":"responded with error XML","hostname":"cdc8a2f93d2f","pid":83}

We are calling the scality with this boto3 code:

    s3 = boto3.resource('s3',
                        aws_access_key_id='accessKey1',
                        aws_secret_access_key='verySecretKey1',
                        endpoint_url='http://s3server:8000')
    s3_client = boto3.client('s3',
                             aws_access_key_id='accessKey1',
                             aws_secret_access_key='verySecretKey1',
                             endpoint_url='http://s3server:8000')

    s3.create_bucket(Bucket='raw-data') # here the exception comes
    bucket = self.s3.Bucket('raw-data')

Answer 1

This issue is quite common. In your config.json file, which you mount in your Docker container, I assume, there is a restEndpoints section, where you must associate a domain name with a default region. What that means is your frontend domain name should be specified in there, matching a default region. Do note that that default region does not prevent you from using other regions: it's just where your buckets will be created if you don't specify otherwise.

In the future, I'd recommend you open an issue directly on the Zenko Forum, as this is where most of the community and core developers are.

Cheers,

Laure