Is it possible to use Spring Cloud AWS for connecting to an on-premise S3 compatible storage?

Question

Is it possible to use Spring Cloud AWS (Spring Cloud AWS Core / Spring Cloud AWS Context) to connect to S3 on-premise (such as Minio/SwiftStack) that support full S3 API?

In short, the URL for S3 service needs to be framed within my application logic instead of the spring cloud AWS building the default based on region.

Answer 1

The application can provide its own AmazonS3 bean, configured to connect to the on-premise S3-compatible storage service.

This works because the Spring Cloud configuration code is configured to not create its own AmazonS3 bean if one is already provided by the application. Therefore, if the application provides its own AmazonS3 bean, that bean will be used as the S3 client instead of the default.

I'm using Spring Cloud AWS to access Amazon's AWS in the running application, and a Dockerized MinIO for integration tests. I've configured my test's AmazonS3 bean based on sample code in MinIO's documentation. This approach should also work for your situation of connecting the actual application to MinIO.

MinIoConfiguration.java

import com.amazonaws.ClientConfiguration;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class MinIoConfiguration {
    @Value("${cloud.aws.credentials.accessKey}")
    private String awsAccessKey;

    @Value("${cloud.aws.credentials.secretKey}")
    private String awsSecretKey;

    @Value("${cloud.aws.region.static}")
    private String awsRegion;

    @Value("${my-app.aws.service-endpoint}")
    private String awsServiceEndpoint;

    @Bean
    public AmazonS3 amazonS3() {
        return AmazonS3Client.builder()
                .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(awsServiceEndpoint, awsRegion))
                .withPathStyleAccessEnabled(true)
                .withClientConfiguration(new ClientConfiguration().withSignerOverride("AWSS3V4SignerType"))
                .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials(awsAccessKey, awsSecretKey)))
                .build();
    }
}

application.yml

cloud:
  aws:
    credentials:
      accessKey: MY_ACCESS_KEY
      secretKey: MY_SECRET_KEY
    region.static: us-east-1

my-app:
  aws:
    service-endpoint: https://example.invalid:9000