Identical request timing out in Java Apache, working fine in postman

Question

I'm trying to register a Office365 Api webhook per official api docs. I tried it in postman, all works as expected.

Java Version: 1.7 (I know...)

I am working with the Play framework version 1.2.7.2

HttpClient: org.apache.http.client.HttpClient

Relevant documentation:

The subscription process goes as follows:

A client app makes a subscription request (POST) for a specific resource. It includes a notification URL, among other properties.

The Outlook notifications service tries to validate the notification URL with the listener service. It includes a validation token in the validation request.

If the listener service successfully validates the URL, it returns a success response within 5 seconds as follows:

Sets the content type in the response header to text\plain. Includes the same validation token in the response body. Returns an HTTP 200 response code. The listener can discard the validation token subsequently. Depending on the URL validation result, the Outlook notifications service sends a response to the client app:

If URL validation was successful, the service creates the subscription with a unique subscription ID and sends the response to the client. If URL validation was unsuccessful, the service sends an error response with an error code and other details. Upon receiving a successful response, the client app stores the subscription ID to correlate future notifications with this subscription.

Postman request:

Both requests intercepted with wireshark:

Postman:

Ý`!Ë2@ʸ1cÊþßV:
ðîPOST /api/v2.0/me/subscriptions HTTP/1.1
Content-Type: application/json
cache-control: no-cache
Postman-Token: a24df796-c49e-4245-a1cf-0949cd6538b6
Authorization: Bearer ###
User-Agent: PostmanRuntime/7.4.0
Accept: */*
Host: localhost:3000
accept-encoding: gzip, deflate
content-length: 430
Connection: keep-alive

{
    "@odata.type":"#Microsoft.OutlookServices.PushSubscription",
    "ChangeType": "Created, Deleted, Updated",
    "ClientState": "some-token",
    "NotificationURL": "https://###/office365/receive.php?subId=5",
    "Resource": "https://outlook.office.com/api/v2.0/me/Calendars('###')/events"
}

(captured in wireshark) (the ### parts are removed info, I made sure that auth and ids match up)

At ###/office365/receive.php lies a php script which just echoes back $_GET['validationtoken']. This works perfectly in Postman.

In Java, I create a request with the same headers and the same body

E/@@5â$¸³zêð-gÄV$
Là¼Là¼POST /api/v2.0/me/subscriptions HTTP/1.1
Accept: */*
Content-Type: application/json
Authorization: Bearer ###
Content-Length: 476
Host: localhost:3000
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.2 (Java/1.7.0_80)
Accept-Encoding: gzip,deflate

{
    "@odata.type":"#Microsoft.OutlookServices.PushSubscription",
    "ChangeType": "Created, Deleted, Updated",
    "ClientState": "1fdb3e372212622e0b7e68abe09e1201a81a8bcc040cce9a6f013f71a09bfbe1",
    "NotificationURL": "https://###/office365/receive.php",
    "Resource": "https://outlook.office.com/api/v2.0/me/Calendars('###')/events"
}

I verified that all (important) parts of the request are exactly the same.

My code in java is a little to complex to share fully here, however the important bits are:

// build url
private final RequestBuilder getRequest (String url) {
    RequestBuilder req  = RequestBuilder.create(getMethod()); // e.g. POST
    req.setUri(overwriteBaseUrl(url) + getPath());

    // headers is a hashmap<String, String>
    for (String key: headers.keySet()) {
        req.setHeader(key, headers.get(key));
    }

    // set body (handle empty cases)
    String body = getBody();
    if (body == null) {
        body = "";
    }

    req.setEntity(new StringEntity(body, "UTF-8"));

    return req;
}

public void send (HttpClient client, Office365Credentials cred, String url) throws IOException, InvalidCrmCredentialsException, MalformedResponseException {
    // creates request (with headers, body, etc)
    RequestBuilder req = getRequest(url);
    // adds auth
    addAuthHeaderToRequest(cred, req);

    // execute the request
    Response response;
    try {
         response = new Response(client.execute(req.build()));
    } catch (ConnectionPoolTimeoutException e) {
        // The 10 second limit has passed
        throw new MalformedResponseException("Response missing (response timed out)!", e);
    }

    // check for 401, not authorized
    if (response.getStatus() == 401) {
        throw new InvalidCrmCredentialsException("401 - Not authorized! " + req.getUri().toString());
    }

    // process response
    processResponse(response);
}

The HttpClient is constructed like this:

    int CONNECTION_TIMEOUT_MS = 10 * 1000; // 10 second timeout
    RequestConfig requestConfig = RequestConfig.custom()
            .setConnectionRequestTimeout(CONNECTION_TIMEOUT_MS)
            .setConnectTimeout(CONNECTION_TIMEOUT_MS)
            .setSocketTimeout(CONNECTION_TIMEOUT_MS)
            .build();

    client = HttpClientBuilder.create().setDefaultRequestConfig(requestConfig).build();

No matter how high I set the limit, I never get a response, the office api. Not even an error response.

TL;DR: My request works fine in postman, exact same request in Java times out (no matter the timeout length)

Answer 1

The problem is, that org.apache.http.client.HttpClient is not thread safe (at least for play framework 1.2.7). Since the play framework reuses threads, my best guess is, that it never received the answer (since the webhook register request takes a little longer than normal requests).

I switched to WS, the http client supplied by the play framework itself. This come with the disadvantage, that it only has limited support for all the http verbs.

The new methods look like this:

private final WS.WSRequest getRequest (String url) {
    WS.WSRequest req = WS.url(overwriteBaseUrl(url) + getPath());

    for (String key: headers.keySet()) {
        req.setHeader(key, headers.get(key));
    }

    String body = getBody();
    if (body == null) {
        body = "";
    }

    req.body(body);

    return req;
}

public void send (WSMockableSender client, Office365Credentials cred, String url) throws IOException, InvalidCrmCredentialsException, MalformedResponseException {

    WS.WSRequest req = getRequest(url);
    addAuthHeaderToRequest(cred, req);

    // execute the request
    WSResponse response;
    response = new WSResponse(client.post(req));

    System.out.println("Response received!");

    // check for 401, not authorized
    if (response.getStatus() == 401) {
        throw new InvalidCrmCredentialsException("401 - Not authorized! " + req.url);
    }

    processResponse(response);
}