Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

deleteObjects using AWS SDK V2?

I'm trying to migrate from AWS SDK V1.x to V2.2. I can't figure out the deleteObjects method though. I've found a bunch of examples - all the same one :-( that doesn't appear to ever use the list of objects to delete (i.e. the list is present, but never set in the DeleteObjectsRequest object - I assume that is where it should be set, but don't see where). How/where do I provide the object list? The examples I find are:

    System.out.println("Deleting objects from S3 bucket: " + bucket_name);
    for (String k : object_keys) {
        System.out.println(" * " + k);
    }

    Region region = Region.US_WEST_2;
    S3Client s3 = S3Client.builder().region(region).build();
    try {
        DeleteObjectsRequest dor = DeleteObjectsRequest.builder()
                .bucket(bucket_name)
                .build();
        s3.deleteObjects(dor);
    } catch (S3Exception e) {
        System.err.println(e.awsErrorDetails().errorMessage());
        System.exit(1);
    }
like image 712
Todd Avatar asked Dec 27 '18 19:12

Todd


People also ask

What is AWS SDK v2?

The AWS SDK for Java provides a Java API for AWS services. Using the SDK, you can easily build Java applications that work with Amazon S3, Amazon EC2, DynamoDB, and more. The AWS SDK for Java 2. x is a major rewrite of the version 1. x code base.

What is the best way to delete multiple objects from S3?

Navigate to the Amazon S3 bucket or folder that contains the objects that you want to delete. Select the check box to the left of the names of the objects that you want to delete. Choose Actions and choose Delete from the list of options that appears. Alternatively, choose Delete from the options in the upper right.

Can I use AWS SDK without credentials?

To make requests to Amazon Web Services, you must supply AWS credentials to the AWS SDK for Java. You can do this in the following ways: Use the default credential provider chain (recommended). Use a specific credential provider or provider chain (or create your own).

What is AWS SDK used for?

The AWS SDK for Java simplifies use of AWS Services by providing a set of libraries that are consistent and familiar for Java developers. It provides support for API lifecycle consideration such as credential management, retries, data marshaling, and serialization.


2 Answers

The previously accepted answer was very helpful even though it wasn't 100% complete. I used it to write the following method. It basically works though I haven't tested its error handling particularly well.

  • An array of String keys is passed in, which are converted into the array of
    ObjectIdentifier's that deleteObjects requires.
  • s3Client and log are assumed to have been initialized elsewhere. Feel free to remove the logging if you don't need it.
  • The method currently returns the number of successfully deletes.

    public int deleteS3Objects(String bucketName, String[] keys) {
    
    List<ObjectIdentifier> objIds = Arrays.stream(keys)
            .map(key -> ObjectIdentifier.builder().key(key).build())
            .collect(Collectors.toList());
    try {
        DeleteObjectsRequest dor = DeleteObjectsRequest.builder()
                .bucket(bucketName)
                .delete(Delete.builder().objects(objIds).build())
                .build();
    
        DeleteObjectsResponse delResp = s3client.deleteObjects(dor);
    
        if (delResp.errors().size() > 0) {
            String err = String.format("%d errors returned while deleting %d objects",
                    delResp.errors().size(), objIds.size());
            log.warn(err);
        }
        if (delResp.deleted().size() < objIds.size()) {
            String err = String.format("%d of %d objects deleted",
                    delResp.deleted().size(), objIds.size());
            log.warn(err);
        }
        return delResp.deleted().size();
    }
    catch(AwsServiceException e) {
        // The call was transmitted successfully, but Amazon S3 couldn't process 
        // it, so it returned an error response.
        log.error("Error received from S3 while attempting to delete objects", e);
    }
    catch(SdkClientException e) {
        // Amazon S3 couldn't be contacted for a response, or the client
        // couldn't parse the response from Amazon S3.
        log.error("Exception occurred while attempting to delete objects from S3", e);
    }
    return 0;
    }
    

(Gratuitous commentary: I find it odd that in AWS SDK v2.3.9, deleteObjects requires Delete.Builder and ObjectIdentifier keys but getObject and putObject accept String keys. Why doesn't DeleteObjectsRequest.Builder just have a keys() method? They haven't officially said the SDK is production-ready AFAIK so some of this may be subject to change.)

like image 62
Bampfer Avatar answered Nov 06 '22 01:11

Bampfer


Looks some more objects are needed to assign the key of the object in s3. This is untested, I put the links to the methods at the end.

System.out.println("Deleting objects from S3 bucket: " + bucket_name);

for (String k : object_keys) {
    System.out.println(" * " + k);
}

Region region = Region.US_WEST_2;
S3Client s3 = S3Client.builder().region(region).build();

try {

   ObjectIdentifier objectToDelete = ObjectIdentifier.Builder()
            .key(KEY_OBJECT_TO_DELETE);

   Delete delete_me Delete.Builder.objects(objectToDelete) 

   DeleteObjectsRequest dor = DeleteObjectsRequest.builder()
            .bucket(bucket_name)
            .delete(delete_me)
            .build();

    s3.deleteObjects(dor);

} catch (S3Exception e) {
    System.err.println(e.awsErrorDetails().errorMessage());
    System.exit(1);
}

Key to delete https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/ObjectIdentifier.html#key--

Delete object https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/ObjectIdentifier.Builder.html

like image 43
strongjz Avatar answered Nov 06 '22 01:11

strongjz