Why is Apache returning a zero content length after 200 seconds to a PHP POST request?

Question

I have PHP script that posts a request to a remote API. If the response takes longer than about 200 seconds to come back, then I just get a content-length of zero in the response. I am trying to figure out why that is happening.

In an attempt to resolve this issue, I have set every conceivable variable in Apache's and PHP's config files set to longer than 300 seconds to combat this, as recommended by the first answer below. The things I have set to 300 seconds:

• Apache timeout
• Apache keep_alive time
• PHP max_reponse_time
• PHP session.cache_expire time
• PHP max_execution_time

Despite that I still consistently get zero content length responses right around the 200-second mark. However if it takes less than 200 seconds the problem does not occur.

Below I describe how our code is set up.

What happens is that crontab runs a shell script on our server, which calls a localhost URI using /usr/bin/curl. The localhost URI is served by Apache and is a PHP file that itself contains the below code, which in turn uses cURL to call out to the remote API. We POST about 10KB of XML and expect to receive about 135KB back, in chunks.

Here is the request code:

        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $this->_xml_url);
        curl_setopt($ch, CURLOPT_POST, true);
        curl_setopt($ch, CURLOPT_VERBOSE, true);
        curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
        curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_str);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        $output = curl_exec($ch);        
        curl_close($ch);

I turned on debugging in our Apache logging and below is what we get. In this example the request was sent at 19:48:00 and the response comes back at 19:51:23, just over 200 seconds later.

* About to connect() to api.asdf.com port 443 (#0)
*   Trying 555.555.555.555... * connected
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSL connection using RC4-SHA
* Server certificate:
*    subject: snip
*    start date: 2014-03-12 10:22:02 GMT
*    expire date: 2015-04-16 12:32:58 GMT
*    subjectAltName: api.asdf.com matched
*    issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign Organization Validation CA - G2
*    SSL certificate verify ok.
> POST /xmlservlet HTTP/1.1

Host: api.asdf.com
Accept: */*
Content-Type: text/xml
Content-Length: 10773
Expect: 100-continue

< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Cache-Control: private
< Content-Type: text/xml
< Server: Microsoft-IIS/7.5
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< Date: Thu, 06 Nov 2014 19:51:23 GMT
< Content-Length: 0
< 

* Connection #0 to host api.asdf.com left intact
* Closing connection #0

I would like to know if there is something wrong with this code or something I may have missed in the server settings that could cause the content length to come back zero after 200 seconds.

Answer 1

Sounds very much like a timeout on your remote API server - try to access it manually e.g. via browser or wget.

Answer 2

and is a PHP file that itself contains the below code, which in turn uses cURL to call out to the remote API. We POST about 10KB of XML and expect to receive about 135KB back, in chunks.

Here is the request code:

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $this->_xml_url);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_VERBOSE, true);
    curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: text/xml'));
    curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_str);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $output = curl_exec($ch);        
    curl_close($ch);

What is very likely to be happening is that the remote server ($this->_xml_url) is actually a load balancer or other front-end block that has a timeout of 200s and calls a backend server.

If the backend server does not answer within the time frame, the front end server closes the backend connection and continues executing (which is arguably the wrong thing to do) and sends to your script a "success" answer with the content it has got.

Which is nothing. Hence content length of zero.

The best you can do is recognize the problem by checking data length or XML conformance, and either retry or notify the user. I'd use a lower timeout than 200s - let's say 180s - in order to be sure that you will be the one raising the error, i.e. the remote server will never hang up on you, you will hang up on him.

Try also notifying the API server maintainers in case there's some way of speeding up the request (a different API? Different encoding? Possibility of caching results? More expensive SLAs? etc.)