Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using S3 Java SDK to talk to S3 compatible storage (minio)

I am trying to use aws-sdk-java AwsS3client to talk to a minio storage. From the CLI I am able to do:

aws --profile=minioplay  --endpoint-url https://play.minio.io:9000 s3 cp logback.xml s3://miniohstest-jixusroqeb --debug

thus using a non default profile and a custom endpoint. Not sure how to do this (would I be able to ?) from the java sdk. I roughly translated the above awscli command to this scala snippet :

val cred = ...
val endpoint = "https://play.minio.io:9000"
val client = AmazonS3ClientBuilder
      .standard()
      .withCredentials(cred)
      .withEndpointConfiguration(
        new EndpointConfiguration(
          endpoint,
          AwsHostNameUtils.parseRegion(endpoint, AmazonS3Client.S3_SERVICE_NAME)
        )
      )
      .build()

Using the above client I am only able to make very simple requests such as :

client.listBuckets().asScala.foreach(println(_))

which works. But when I try to do something advanced such as :

val listRequest = new ListObjectsRequest()
      .withBucketName("miniohstest-jixusroqeb")
      //.withPrefix(r.getURI.getPath)
      //.withDelimiter(delimiter)

val res = client.listObjects(listRequest)
res.getObjectSummaries.forEach(x => println(x.getKey))

it throws the following exception :

Exception in thread "main" com.amazonaws.SdkClientException: Unable to execute HTTP request: miniohstest-jixusroqeb.play.minio.io
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1114)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1064)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)

What am I doing wrong?

like image 223
takirala Avatar asked Mar 17 '18 04:03

takirala


People also ask

How does MinIO work with S3?

Feature Bucket & Object Versioning With MinIO, objects are independently versioned following Amazon's S3 structure/implementation. MinIO assigns a unique ID to each version of a given object - applications can specify a version ID at any time to access the point-in-time snapshot of that object.

Is MinIO same as S3?

So, What Is Minio? The short and simplified answer is “It's like Amazon S3, but hosted locally.” Minio is an object storage server that implements the same public API as Amazon S3. This means that applications that can be configured to talk to Amazon S3 can also be configured to talk to Minio.

Can I use AWS CLI with MinIO?

AWS CLI with MinIO Server AWS CLI is a unified tool to manage AWS services. It is frequently the tool used to transfer data in and out of AWS S3. It works with any S3 compatible cloud storage service. In this recipe we will learn how to configure and use AWS CLI to manage data with MinIO Server.


1 Answers

I resolved this by setting withPathStyleAccessEnabled(true).

like image 193
takirala Avatar answered Oct 07 '22 21:10

takirala