How to forward large files with RestTemplate?

Question

I have a web service call through which zip files can be uploaded. The files are then forwarded to another service for storage, unzipping, etc. For now the file is stored on the file system, then a FileSystemResource is built.

Resource zipFile = new FileSystemResource(tempFile.getAbsolutePath()); 

I could use a ByteStreamResource in order to save time(the saving of the file on disk is not needed before forwarding) but for that i need to build a byte array. In case of large files I will get an "OutOfMemory : java heap space" error.

ByteArrayResource r = new ByteArrayResource(inputStream.getBytes()); 

Any solutions to forwarding files without getting an OutOfMemory error using RestTemplate?

Answer 1

Edit: The other answers are better (use Resource) https://stackoverflow.com/a/36226006/116509

My original answer:

You can use execute for this kind of low-level operation. In this snippet I've used Commons IO's copy method to copy the input stream. You would need to customize the HttpMessageConverterExtractor for the kind of response you're expecting.

final InputStream fis = new FileInputStream(new File("c:\\autoexec.bat")); // or whatever final RequestCallback requestCallback = new RequestCallback() {      @Override     public void doWithRequest(final ClientHttpRequest request) throws IOException {         request.getHeaders().add("Content-type", "application/octet-stream");         IOUtils.copy(fis, request.getBody());      } }; final RestTemplate restTemplate = new RestTemplate(); SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory(); requestFactory.setBufferRequestBody(false);      restTemplate.setRequestFactory(requestFactory);      final HttpMessageConverterExtractor<String> responseExtractor =     new HttpMessageConverterExtractor<String>(String.class, restTemplate.getMessageConverters()); restTemplate.execute("http://localhost:4000", HttpMethod.POST, requestCallback, responseExtractor); 

(Thanks to Baz for pointing out you need to call setBufferRequestBody(false) or it will defeat the point)

Answer 2

The only part of @artbristol's answer you really need is this (which you can set up as a RestTemplate Spring bean):

final RestTemplate restTemplate = new RestTemplate(); SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory(); requestFactory.setBufferRequestBody(false);      restTemplate.setRequestFactory(requestFactory);      

After that, I think just using a FileSystemResource as your request body will do the right thing.

I've also used an InputStreamResource successfully this way, for cases where you already have the data as an InputStream and don't need to consume it multiple times.

In my case, we had gzipped our files and wrapped a GZipInputStream in an InputStreamResource.