Jersey resource receiving duplicate requests from Jersey client

Question

We recently upgraded from Jersey 1.x to Jersey 2.22.1 on both Server and Clients. We are now seeing intermittently Jersey will make/receive two requests.

• By intermittent I mean every 1 out of several thousand requests.
• We never experienced this issue using Jersey 1.x.
• It is not clear to me if this is an issue on the client or server side.
• On the client side the log only shows a single POST request and response (see snippet below)
• On the server side the log shows two POST requests and responses (see snippet below)

I'm able to reproduce it by looping many thousands of times over this client POST request. Each request sends a unique 'name' which gets persisted on the server. I know we have received a duplicate request when i get a unique constraint violation trying to persist the same 'name' twice. I ruled out other sections of the code because the log confirms Jersey is receiving two POST requests for the same 'name'

I've enabled trace logging in org.glassfish package on the server and registered LoggingFilter() on the client.

Client show only 1 POST request and response:

Jun 21, 2016 6:02:51 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 8291 * Sending client request on thread main
8291 > POST http://localhost:9797/my-webapp/v1/data-feeds/
8291 > Accept: application/json
8291 > Content-Type: application/json

Jun 21, 2016 6:02:51 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 8291 * Client response received on thread main
8291 < 200
8291 < Cache-Control: no-cache, no-store, max-age=0, must-revalidate
8291 < Content-Length: 181
8291 < Content-Type: application/json
8291 < Date: Wed, 22 Jun 2016 00:02:51 GMT
8291 < Expires: 0
8291 < Pragma: no-cache
8291 < Server: Apache-Coyote/1.1
8291 < Set-Cookie: JSESSIONID=CFF556E7FCDB5B1F644BA04603364DFD; Path=/my-webapp/; HttpOnly
8291 < X-Content-Type-Options: nosniff
8291 < X-Frame-Options: DENY
8291 < X-XSS-Protection: 1; mode=block

Server shows two POSTS for the same 'name':

Jun 21, 2016 6:02:51 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 8293 * Server has received a request on thread http-bio-9797-exec-21
8293 > POST http://localhost:9797/my-webapp/v1/data-feeds/
8293 > accept: application/json
8293 > authorization: Basic YWRtaW46bmltZGE=
8293 > connection: keep-alive
8293 > content-length: 181
8293 > content-type: application/json
8293 > host: localhost:9797
8293 > user-agent: Jersey/2.22.1 (HttpUrlConnection 1.8.0_31)

2016-06-21 18:02:51,964 [INFO] [c.m.c.r.r.MyResource] Received POST request /data-feeds with args [FeedData{name='pool4146'}]
Jun 21, 2016 6:02:51 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 8294 * Server has received a request on thread http-bio-9797-exec-97
8294 > POST http://localhost:9797/my-webapp/v1/data-feeds/
8294 > accept: application/json
8294 > authorization: Basic YWRtaW46bmltZGE=
8294 > connection: keep-alive
8294 > content-length: 181
8294 > content-type: application/json
8294 > host: localhost:9797
8294 > user-agent: Jersey/2.22.1 (HttpUrlConnection 1.8.0_31)

2016-06-21 18:02:51,978 [INFO] [c.m.c.r.r.MyResource] Received POST request /data-feeds with args [FeedData{name='pool4146'}]
Jun 21, 2016 6:02:51 PM org.glassfish.jersey.filter.LoggingFilter log

INFO: 8293 * Server responded with a response on thread http-bio-9797-exec-21
8293 < 200
8293 < Content-Type: application/json

Jun 21, 2016 6:02:51 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 8294 * Server responded with a response on thread http-bio-9797-exec-97
8294 < 200
8294 < Content-Type: application/json

Let me know if there is any other config info that might be relevant here. We are using Tomcat 7.x and Jackson for serialization/deserialization

Answer 1

As the solution was not part of the Answer from Bastian Baist, i'll post it here. It seems that Jersey bug tracker was moved to GitHub, so for reference this is the new and working link: https://github.com/jersey/jersey/issues/3526

To fix this issue one may add the following to jersey/core-client/src/main/java/org/glassfish/jersey/client/authentication/HttpAuthenticationFilter.java:

static boolean repeatRequest(ClientRequestContext request, ClientResponseContext response, String newAuthorizationHeader) throws IOException {
// If the failed response has an entity stream, close it. We must do this to avoid leaking resources
// when we replace the entity stream of the failed response with that of the repeated response (see below).
// Notice that by closing the entity stream before sending the repeated request we allow resources allocated
// to the failed request (e.g., the HTTP connection) to be reused, if possible, for the repeated request.
if (response.hasEntity()) {
    response.getEntityStream().close();
    response.setEntityStream(null);
}

Client client = request.getClient();

And remove this line of code:

 Client client = ClientBuilder.newClient(request.getConfiguration());

I'ts annoying that jersey maintainers seems to not fixing this issues.