Downloading AWS S3 file as a stream in Spring boot

Question

I want to expose an API to download a S3 bucket file content as stream to its consumers. The API URL is like /downloadfile/** which is GET request.

1. What should be my return type right now I tried with accept header= application/octet-stream which didn't work.
2. I don't want to write the content of file to any file and send it. It should be returned as a stream that's it.

Here is the controller pseudo code I wrote till now which is giving me 406 error all the time.

 @GetMapping(value = "/downloadfile/**", produces = { MediaType.APPLICATION_OCTET_STREAM_VALUE })
    public ResponseEntity<Object> downloadFile(HttpServletRequest request) {
       //reads the content from S3 bucket and returns a S3ObjectInputStream
       S3ObjectInputStream object = null;
       object = publishAmazonS3.getObject("12345bucket", "/logs/file1.log").getObjectContent();
       return object
    }

Any suggestions here on the way of doing this and what I am doing wrong?

Answer 1

I was able to download the file as a stream by using StreamingResponseBody class from Spring.

Here is the code I used:

    @GetMapping(value = "/downloadfile/**", produces = { MediaType.APPLICATION_OCTET_STREAM_VALUE })
    public ResponseEntity<StreamingResponseBody> downloadFile(HttpServletRequest request) {
       //reads the content from S3 bucket and returns a S3ObjectInputStream
       S3Object object = publishAmazonS3.getObject("12345bucket", "/logs/file1.log");
       S3ObjectInputStream finalObject = object.getObjectContent();

        final StreamingResponseBody body = outputStream -> {
            int numberOfBytesToWrite = 0;
            byte[] data = new byte[1024];
            while ((numberOfBytesToWrite = finalObject.read(data, 0, data.length)) != -1) {
                System.out.println("Writing some bytes..");
                outputStream.write(data, 0, numberOfBytesToWrite);
            }
            finalObject.close();
        };
        return new ResponseEntity<>(body, HttpStatus.OK);
    }

The way to test the streaming is done correctly or not is to have a file of around 400mb to download. Reduce your Xmx to 256mb by passing the in the vm options. Now, compare the download functionality with and without using StreamingResponseBody, you will get OutofMemoryError when using the conventional OutputStreams for writing the content