Async timeout downloading a large file using StreamingResponseBody on Spring Boot

Question

I'm trying to expose a REST Service which makes available to download a large file streaming, without keeping in memory first. Also I need this to support async calls, if (at least) two users on the same time call this URL should be able both of them to download it. Application is set up with Spring Boot.
This is what I have on Controller:

@RestController
public class MyController {

   private MyService service;

   @Autowired
   public MyController(MyService service) {
       this.service = service;
   }

   @RequestMapping(
        value = "download",
        method = RequestMethod.GET,
        produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
   public ResponseEntity<StreamingResponseBody> downloadAsync() throws IOException {

       StreamingResponseBody responseBody = outputStream -> {
           service.download(outputStream);
           outputStream.close();
       };

       return ResponseEntity.ok(responseBody);
    }
}

This is what I have on Service (download URL is just a sample to test this behavior):

@Service
public class MyService {

    private RestTemplate restTemplate;

    @Autowired
    public MyService(RestTemplate restTemplate) {
        this.restTemplate = restTemplate;
    }

    public void download(OutputStream outputStream) {

        ResponseExtractor<Void> responseExtractor = clientHttpResponse -> {
            InputStream inputStream = clientHttpResponse.getBody();
            StreamUtils.copy(inputStream, outputStream);
            return null;
        };

        restTemplate.execute("http://download.thinkbroadband.com/1GB.zip",
            HttpMethod.GET,
            clientHttpRequest -> {},
            responseExtractor);
    }
}

In application.yml among others, I have these properties, nothing fancy at all:

server:
  port: 9999
  context-path: /rest

And this is the JavaConfig file:

@Configuration
public class ApplicationConfig {
    @Bean
    public RestTemplate restTemplate() {
        ClientHttpRequestFactory requestFactory =
            new HttpComponentsClientHttpRequestFactory(HttpClients.createDefault());

        RestTemplate restTemplate = new RestTemplate(requestFactory);
        restTemplate.setErrorHandler(new ClientErrorHandler());
        return restTemplate;
    }
}

When I call this endpoint localhost:9999/rest/download download starts and downloads some MBs but after some time, it stops and this is what gets shown on my console:

2017-03-18 17:11:54.808 INFO  --- [nio-9999-exec-1] o.a.c.c.C.[.[.[/rest]                    : Initializing Spring FrameworkServlet 'dispatcherServlet'
2017-03-18 17:11:54.811 INFO  --- [nio-9999-exec-1] o.s.w.s.DispatcherServlet                : FrameworkServlet 'dispatcherServlet': initialization started
2017-03-18 17:11:54.895 INFO  --- [nio-9999-exec-1] o.s.w.s.DispatcherServlet                : FrameworkServlet 'dispatcherServlet': initialization completed in 84 ms
2017-03-18 17:12:25.334 ERROR --- [nio-9999-exec-2] .w.s.m.s.DefaultHandlerExceptionResolver : Async timeout for GET [/rest/download]
2017-03-18 17:12:25.335 WARN  --- [nio-9999-exec-2] .w.s.m.s.DefaultHandlerExceptionResolver : Resolved exception caused by Handler execution: org.springframework.web.context.request.async.AsyncRequestTimeoutException
2017-03-18 17:12:25.366 INFO  --- [nio-9999-exec-2] o.a.c.c.CoyoteAdapter                    : Encountered a non-recycled response and recycled it forcedly.
org.apache.catalina.connector.CoyoteAdapter$RecycleRequiredException: null
    at org.apache.catalina.connector.CoyoteAdapter.checkRecycled(CoyoteAdapter.java:494) [tomcat-embed-core-8.5.11.jar:8.5.11]
    at org.apache.coyote.http11.Http11Processor.recycle(Http11Processor.java:1627) [tomcat-embed-core-8.5.11.jar:8.5.11]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.release(AbstractProtocol.java:977) [tomcat-embed-core-8.5.11.jar:8.5.11]
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:869) [tomcat-embed-core-8.5.11.jar:8.5.11]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1434) [tomcat-embed-core-8.5.11.jar:8.5.11]
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.11.jar:8.5.11]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_60]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_60]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.11.jar:8.5.11]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]

Can anyone help, please ? Thanks in advance

Answer 1

If you are encountering this issue using Spring-Boot, it is enough to set the following property to a higher value - for example:

spring:
  mvc:
    async:
      request-timeout: 3600000

or

spring.mvc.async.request-timeout = 3600000

Answer 2

It seems that you run into timeout issues on your async task executor. You can configure the desired timeout (and other settings) with a WebMvcConfigurerAdapter. This code should help resolve this problem. Be sure to replace the ellipsis (...) with the desired values.

This example also registers an interceptor that gets called when there's a timeout in case you want some special handling.

@Configuration
@EnableAsync
@EnableScheduling
public class AsyncConfiguration implements AsyncConfigurer {

    private final Logger log = LoggerFactory.getLogger(AsyncConfiguration.class);

    @Override
    @Bean(name = "taskExecutor")
    public AsyncTaskExecutor getAsyncExecutor() {
        log.debug("Creating Async Task Executor");
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(...);
        executor.setMaxPoolSize(...);
        executor.setQueueCapacity(...);
        executor.setThreadNamePrefix(...);
        return executor;
    }

    @Override
    public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
        return new SimpleAsyncUncaughtExceptionHandler();
    }

    /** Configure async support for Spring MVC. */
    @Bean
    public WebMvcConfigurerAdapter webMvcConfigurerAdapter(AsyncTaskExecutor taskExecutor, CallableProcessingInterceptor callableProcessingInterceptor) {
        return new WebMvcConfigurerAdapter() {
            @Override
            public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
                configurer.setDefaultTimeout(...)
                    .setTaskExecutor(taskExecutor);
                configurer.registerCallableInterceptors(callableProcessingInterceptor);
                super.configureAsyncSupport(configurer);
            }
        };
    }

    @Bean
    public CallableProcessingInterceptor callableProcessingInterceptor() {
        return new TimeoutCallableProcessingInterceptor() {
            @Override
            public <T> Object handleTimeout(NativeWebRequest request, Callable<T> task) throws Exception {
                log.error("timeout!");
                return super.handleTimeout(request, task);
            }
        };
    }
}