How to reduce java heap usage when reading >50mb files?

Question

I am developing a webapplication which reads very large files > 50MB and display it. The Java Spring backend will read these files and its content with CXF. My problem is, after it reads a 50 megabyte file, the size of used heap is increasing by 500 megabytes. I read this fils as a String and this String will be sent to frontend. Is there any idea, tricks how can I reduce the Java heap usage? I tried nio, spring's Resource class, but nothing helped.

Answer 1

A dirty way to do this is to have the Spring @Controller method accept an OutputStream or Writer argument—the framework will supply the raw output stream of the HTTP response and you can write directly into it. This sidesteps all the nice logic of content type management etc.

A better way is to define a custom type which will be returned from the controller method and a matching HttpMessageConverter which will (for example) use the information in that object to open the appropriate file and write its contents into the output stream.

In both cases you will not read the file into RAM; you'll use a single, fixed-size buffer to transfer the data directly from the disk to the HTTP response.