How To Stream Chunked Response With Spring Boot @RestController

Question

I have spent like a day on this and I am unable to find a solution that works. In our application we have a couple of endpoints that can return large responses. I have been trying to find a mechanism that allows us to stream the response as we process the result of a database query. The main goals are to limit peak memory usage (not need the entire response in memory) on the service side and to minimize the time to first byte of response (the client system has a timeout if the response doesn't start to come within the specified time - 10 minutes). I'm really surprised this is so hard.

I found StreamingResponseBody and it seemed close to what we wanted, although we don't really need the asynchronous aspect, we only want to be able to start streaming the response as we process the query result. I have tried other approaches as well, like annotating with @ResponseBody, returning void, and adding a parameter of OutputStream, but that didn't work because the passed OutputStream was basically just a CachingOutputStream that buffered the entire result. Here is what I have now...

Resource Method:

@GetMapping(value = "/catalog/features")
public StreamingResponseBody findFeatures(                                      
        @RequestParam("provider-name") String providerName,
        @RequestParam(name = "category", required = false) String category,
        @RequestParam("date") String date,
        @RequestParam(value = "version-state", defaultValue = "*") String versionState) {

    CatalogVersionState catalogVersionState = getCatalogVersionState(versionState);

    log.info("GET - Starting DB query...");
    final List<Feature> features 
        = featureService.findFeatures(providerName, 
                                      category, 
                                      ZonedDateTime.parse(date), 
                                      catalogVersionState);
    log.info("GET - Query done!");

    return new StreamingResponseBody() {
        @Override
        public void writeTo(OutputStream outputStream) throws IOException {
            log.info("GET - Transforming DTOs");
            JsonFactory jsonFactory = new JsonFactory();
            JsonGenerator jsonGenerator = jsonFactory.createGenerator(outputStream);
            Map<Class<?>, JsonSerializer<?>> serializerMap = new HashMap<>();
            serializerMap.put(DetailDataWrapper.class, new DetailDataWrapperSerializer());
            serializerMap.put(ZonedDateTime.class, new ZonedDateTimeSerializer());
            ObjectMapper jsonMapper =  Jackson2ObjectMapperBuilder.json()
                .serializersByType(serializerMap)
                .deserializerByType(ZonedDateTime.class, new ZonedDateTimeDeserializer())
                .build();
            jsonGenerator.writeStartArray();
            for (Feature feature : features) {
                FeatureDto dto = FeatureMapper.MAPPER.featureToFeatureDto(feature);
                jsonMapper.writeValue(jsonGenerator, dto);
                jsonGenerator.flush();
            }
            jsonGenerator.writeEndArray();
            log.info("GET - DTO transformation done!");
        }
    };
}

Async Configuration:

@Configuration
@EnableAsync
@EnableScheduling
public class ProductCatalogStreamingConfig extends WebMvcConfigurerAdapter {

    private final Logger log = LoggerFactory.getLogger(ProductCatalogStreamingConfig.class);

    @Override
    public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
        configurer.setDefaultTimeout(360000).setTaskExecutor(getAsyncExecutor());
        configurer.registerCallableInterceptors(callableProcessingInterceptor());
    }

    @Bean(name = "taskExecutor")
    public AsyncTaskExecutor getAsyncExecutor() {
        log.debug("Creating Async Task Executor");
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(2);
        executor.setMaxPoolSize(5);
        executor.setQueueCapacity(25);
        executor.setThreadNamePrefix("AsyncStreaming-");
        return executor;
    }

    @Bean
    public CallableProcessingInterceptor callableProcessingInterceptor() {
        return new TimeoutCallableProcessingInterceptor() {
            @Override
            public <T> Object handleTimeout(NativeWebRequest request, Callable<T> task) throws 
Exception {
                log.error("timeout!");
                return super.handleTimeout(request, task);
            }
        };
    }
}

I was expecting that the client would start seeing the response as soon as StreamingResponseBody.writeTo() was called and that the response headers would include

Content-Encoding: chunked

but not

Content-Length: xxxx

Instead, I don't see any response at the client until StreamingResponseBody.writeTo() has returned and the response includes the Content-Length. (but not Content-Encoding)

My question is, What is the secret sauce that tells Spring to send a chunked response while I'm writing to OutputStream in writeTo() and not cache the entire payload and send it only at the end? Ironically I have found posts that want to know how to disable chunked encoding, but nothing about enabling it.

Answer 1

It turns out the code above does exactly what we were seeking. The behavior we observed was not due to anything in the way Spring has implemented these features, it was caused by a company specific starter that installed a servlet filter that interfered with the normal Spring behavior. This filter wrapped the HttpServletResponse OutputStream and that is why we observed the CachingOutputStream noted in the question. After removing the starter, the above code behaved exactly as we hoped and we are re-implementing the servlet filter in a way that will not interfere with this behavior.