java.net.HttpRetryException: cannot retry due to server authentication, in streaming mode

Question

We have two parts in our app:
Server - provide REST services
Client - consume them via Spring restTemplate

In addition to the HTTP status our server returns an HTTP body with JSON that describe error in detail. So, I've added custom Error handler to restTemplate to treat some error coded as non errors - it helps parse HTTP body very well.

But I get an exception via parsing of the HTTP body in the case of an HTTP/1.1 401 Unauthorized. All other error codes are handled fine(400, 402, etc. ) We are using plain server logic that sends HTTP response in the case of an error, no special rules for different types of an error:

writeErrorToResponse(int status, String errMsg, HttpServletResponse resp) throws IOException {
        response.setStatus(status);
        response.setContentType(MediaType.APPLICATION_JSON_VALUE);
        String message = String.format("{\"error\":\"%s\"}", StringUtils.escapeJson(errMsg));
        resp.getWriter().println(message);
    }

But on client only HTTP/1.1 401 throws exception - "java.net.HttpRetryException: cannot retry due to server authentication, in streaming mode"

I've made some debugging and see that the cause of the problem is code in SimpleClientHttpResponse:

HttpURLConnection.getInputStream()

Tracing with Fiddler have these next responses: Message is parsed correct on the client:

HTTP/1.1 402 Payment Required
X-Powered-By: Servlet/3.0
Content-Type: application/json
Content-Language: en-GB
Content-Length: 55
Connection: Close
Date: Sat, 25 May 2013 10:10:44 GMT
Server: WebSphere Application Server/8.0

{"error":"I cant find that user.  Please try again."}

And message that is cause of exception:

HTTP/1.1 401 Unauthorized
X-Powered-By: Servlet/3.0
Content-Type: application/json
Content-Language: en-GB
Content-Length: 55
Date: Sat, 25 May 2013 11:00:21 GMT
Server: WebSphere Application Server/8.0

{"error":"I cant find that user.  Please try again."}

What could be the cause of java.net.HttpRetryException in this situation?

In addition: Some times ago this mechanism worked fine. But since we have changed a lot of code in app.

Answer 1

I faced the same issue when using SimpleClientHttpRequestFactory. Solved it by setting

SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setOutputStreaming(false);
return requestFactory;

The problem is due to chunking and subsequent retry mechanism incase of authentication.

You can also disable chunks using HttpClientPolicy

Answer 2

This issue is reported in Spring Framework.

Reference: https://jira.spring.io/browse/SPR-9367

Copying from Reference: It is very difficult (impossible) to handle a 401 response in the RestTemplate with default settings. In fact it is possible, but you have to supply an error handler and a request factory. The error handler was obvious, but the problem is that the default request factory uses java.net which can throw HttpRetryException when you try to look at the status code of the response (despite it being obviously available). The solution is to use HttpComponentsClientHttpRequestFactory. E.g.

template.setRequestFactory(new HttpComponentsClientHttpRequestFactory());
template.setErrorHandler(new DefaultResponseErrorHandler() {
    public boolean hasError(ClientHttpResponse response) throws IOException {
        HttpStatus statusCode = response.getStatusCode();
        return statusCode.series() == HttpStatus.Series.SERVER_ERROR;
    }
});

HttpComponentsClientHttpRequestFactory requires below dependency. Add below dependency in your POM file:

<dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
</dependency>

Answer 3

I was having the exact same issue using Spring Boot 2.2.4 and a TestRestTemplate. It only failed on 401's. Simply adding the latest version of Http Client (4.5.11) in a test scope fixed the problem. No further configuration was necessary.

Answer 4

I think passing a configured HttpComponentsClientHttpRequestFactory to your RestTemplate would help. Here you might find the solution on how to do it.

The reason that I used such development solution was to handle the response messages in the body of Http error responses like 401, 404, and so on, and also to let the application be deployed on real-server side environment.