AWS S3 Event notification using Lambda function in Java

Question

I am trying to use Lambda function for S3 Put event notification. My Lambda function should be called once I put/add any new JSON file in my S3 bucket. The challenge I have is there are not enough documents for this to implement such Lambda function in Java. Most of doc I found are for Node.js

I want, my Lambda function should be called and then inside that Lambda function, I want to consume that added json and then send that JSON to AWS ES Service.

But what all classes I should use for this? Anyone has any idea about this? S3 abd ES are all setup and running. The auto generated code for lambda is `

@Override
public Object handleRequest(S3Event input, Context context) {
    context.getLogger().log("Input: " + input);

    // TODO: implement your handler
    return null;
}

What next??

Answer 1

Handling S3 events in Lambda can be done, but you have to keep in mind, the the S3Event object only transports the reference to the object and not the object itself. To get to the actual object you have to invoke the AWS SDK yourself. Requesting a S3 Object within a lambda function would look like this:

public Object handleRequest(S3Event input, Context context) {
    AmazonS3Client s3Client = new AmazonS3Client(new DefaultAWSCredentialsProviderChain());        

    for (S3EventNotificationRecord record : input.getRecords()) {
        String s3Key = record.getS3().getObject().getKey();
        String s3Bucket = record.getS3().getBucket().getName();
        context.getLogger().log("found id: " + s3Bucket+" "+s3Key);
        // retrieve s3 object
        S3Object object = s3Client.getObject(new GetObjectRequest(s3Bucket, s3Key));
        InputStream objectData = object.getObjectContent();
        //insert object into elasticsearch
    }        
    return null;
}

Now the rather difficult part to insert this object into ElasticSearch. Sadly the AWS SDK does not provide any functions for this. The default approach would be to do a REST call against the AWS ES endpoint. There are various samples out their on how to proceed with calling an ElasticSearch instance.

Some people seem to go with the following project:

Jest - Elasticsearch Java Rest Client