Apache, PHP Invalid Content Length

Question

I have a PHP (PHP 5.3.2) application running on Apache (2.2.14) on Linux server. More then once in its history user started to see in their browsers the following message: "Request entity too large". First google search suggested that it happens when an uploaded file is too big or when cookies are too big. But it is not happening on file upload, only at logging in and always at the login page. User enters this page just through a normal POST form on a page. And the cookies are only containing session id.

Furthermore, apache is logging "Invalid Content-Length". This is being logged before any of my code executes. $_POST is empty despite it should contain some data. When I added the following line:

error_log("Content-Length: ".$_SERVER['CONTENT_LENGTH']);

it says

Content-Length: 32, 32

which seems to be indeed invalid. Changing browser or resetting user PC does not seem to help. Everything just started to work the next day.

Where can the problem lay? Is it something with Apache, PHP, user's network settings? If I need to add some more information, please let me know.

Answer 1

In the past days we encountered "Invalid Content-Length" messages logged by Apache in error_log. Tracked the issue to some visitor from just a few specific IP classes, for example Movistar Argentina: 200.81.44.201 Further checking the request with tcpdump / whireshark we observed that the POST request contained some

duplicate Headers, namely Content-Length and Content-Type

The real issue is with Content-Length, because Apache treat multiple variables, by concatenating them, for example:

Content-Length: 51
Content-Length: 51

The result will be:

Content-Length: 51, 51

Then Apache try to convert the string "51, 51" into an integer which is not possible. In newer Apache > 2.2.17, or with backported patches Apache logs an error: 413 Request Entity Too Large, with older Apache, the process just hungs consuming a lot of CPU times. Behind that IP class we see more distinct clients, so maybe they are 3G gateway proxy servers wrongly configured, because they duplicate that headers. So we find a solution to configure Apache to rewrite the Request with just one Content-Length and one Content-Type Headers: Here is my code for this, using mod_headers, and SetEnvIfNoCase inserted in httpd.conf:

SetEnvIfNoCase Content-Length (.*), MyContentLength=$1
SetEnvIfNoCase Content-Type (.*), MyContentType=$1
RequestHeader set Content-Length: "%{MyContentLength}e" env=MyContentLength
RequestHeader set Content-Type: "%{MyContentType}e" env=MyContentType

You are free to test and use this code in order to filter out duplicate Content-Length HTTP Header variables.

Answer 2

The easiest way to investigate issues like this is to use TCPDump to capture the packets being sent between the client and the server to a file, and then use Wireshark to view the packets as a TCP stream.

An example of the command to capture the packets would be.

tcpdump -vvv -A -i eth0 'host 123.123.10.12 && port 80' -w eth0.dat

As it's your login page, you will probably need to decode the TCP stream as it should be behind HTTPS - which can be done using the instructions from http://segfault.in/2010/11/decrypt-https-traffic-using-wireshark-and-key-file/

Plasmid87 is almost certainly correct though - it sounds like one of the headers is just growing massively and eventually becoming larger than the Apache LimitRequestFieldSize.

So long as it's growing slowly rather than all at once, you could investigate it by checking the global variables.

function checkVars(){
    $varsToCheck = array($GLOBALS, $_SERVER, $_GET, $_POST, $_FILES, $_REQUEST, $_SESSION, $_ENV, $_COOKIE);

    foreach($varToCheck as $var){
        foreach($var as $key => $value){

            if(count($key) > 1024){
                logToFile("Found ridiculous key: ".$key." value is ".$value);
            }
            if(count($value) > 1024){
                logToFile("Found ridiculous value for key: ".$key." value is ".$value);
            }
        }
    }
}