How to set S3 path style in Clojure using Amazonica library?

Question

I'm a Ruby dev moving over to Clojure and I'm having trouble understanding how to translate the following Java call into Clojure based on the conventions used in the Clojure library Amazonica.

AmazonS3 client = new AmazonS3Client(credentials);
client.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(true));

The code I have at the moment is:

(ns spurious-aws-sdk-helper.core
  (:use [amazonica.aws.s3]])
  (:require [amazonica.core :refer [ex->map]]))

(def credentials {:access-key "development_access"
                  :secret-key "development_secret"
                  :endpoint "s3.spurious.localhost:49154"
                  :client-config {:protocol "http"}})

(try
  (amazonica.aws.s3/set-s3client-options {:path-style-access true})
  (create-bucket credentials "testing")
  (catch Exception e
    (clojure.pprint/write (ex->map e))))

But I'm getting the following error:

com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Unable to execute HTTP request: testing.s3.spurious.localhost
java.net.UnknownHostException: testing.s3.spurious.localhost

This doesn't look to be correct as it's prefixing the bucket name (testing) onto the hostname. Where as I need the SDK to talk to our local (fake) S3 service (s3.spurious.localhost:49154) using path style.

e.g. like http://s3.spurious.localhost:49154/testing

I think it's because I'm not translating the Java code correctly...

(amazonica.aws.s3/set-s3client-options {:path-style-access true})

...this is passing a map to set-s3client-options rather than what it should be, which is passing the result of calling withPathStyleAccess(true) on a new instance of S3ClientOptions. But I don't know how to do that here?

Any help would be greatly appreciated.

UPDATE

Here is the latest version of the code (which still doesn't work)...

(ns spurious-aws-sdk-helper.core
  (:use [amazonica.aws.s3])
  (:require [amazonica.core :refer [ex->map]]))

(def credentials {:access-key "development_access"
                  :secret-key "development_secret"
                  :endpoint "s3.spurious.localhost:49154"
                  :client-config {:protocol "http"}})

(try
  (amazonica.aws.s3/set-s3client-options 
    (. (com.amazonaws.services.s3.S3ClientOptions.) setPathStyleAccess true))
  (create-bucket credentials "testing")
  (catch Exception e
    (clojure.pprint/write (ex->map e))))

Answer 1

Your first call to set up the S3ClientOptions is the correct one.

I think you're confusing what that setting does. It changes the pre-signed URL generation from using the Virtual Hosted Style https://my_bucket_name.s3.amazonaws.com/my_key to the Path Style https://s3.amazonaws.com/my_bucket_name/my_key

This is necessary if your bucket name contains dots, as the virtual hosted style breaks Amazon's SSL certificate identification (i.e. it only supports the wildcard *.s3.amazonaws.com)

So if you're generating pre-signed URLs on a bucket name containing dots in Clojure e.g.

(s3/generate-presigned-url bucket key (-> 6 hours from-now)))

You will first need to set the path style using

(amazonica.aws.s3/set-s3client-options {:path-style-access true})

Answer 2

Thanks to @stukennedy for responding to the open question. I should have come back long ago and updated this space with the actual solution which I managed to figure out and implement ~8months ago (Feb 2nd 2015):

(ns spurious-aws-sdk-helper.s3
  (:use [amazonica.aws.s3])
  (:require [spurious-aws-sdk-helper.utils :refer [endpoint cred]]))

(defn resource [type]
  (endpoint type :spurious-s3))

(defn setup
  ([type]
   (set-s3client-options (cred (resource type)) :path-style-access true))
  ([type name]
   (try
     (let [credentials (cred (resource type))]
       (set-s3client-options credentials :path-style-access true)
       (create-bucket credentials name))
     (catch Exception e
       (prn "S3 Error: chances are you're creating a bucket that already exists")))))

You can find the full details here: https://github.com/Integralist/spurious-clojure-aws-sdk-helper if you need more context

If I remember correctly, I needed to pass credentials through to set-s3client-options (without them it wouldn't work)