Very long json response stops and sends HTTP headers as text then continues

Question

I am totally lost with this problem.

I have an ajax query which gets a json response. The query works fine in most situations however seems to stumble when the json response is very large.

The problem is the response ends up in the form:

...est":"test length"}]]}
HTTP/1.1 200 OK
Date: Wed, 21 Sep 2011 17:10:32 GMT
Server: Apache/2.2.11 (Win32) mod_ssl/2.2.11 OpenSSL/0.9.8k PHP/5.3.0
X-Powered-By: PHP/5.3.0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Keep-Alive: timeout=5, max=90
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html

5b03d
{"ResultsInfo":{"RequestID":"131661886010","FeedCompletion":{"0":"100"}},"ResultsData":[[{"test":"test length"},{"test":"test length"},

...0

The ... represent more of the same "{"test":"test length"}," string

So the reponse seems to be in the form:

• Last part of the data
• http response header printed out in the body
• The characters '5b03d'
• First part of the data
• The character '0'

There is NOT an EXACT length of response that this occurs at however it is fine at 360791 characters but not at 372797 characters.

I am using the Yii PHP framework, but have searched far and wide and not seen anything in there forums.

It seems to me like the webserver is separating the response into several pieces or timing out and starting again.

Or perhaps there is a max size of return?

EDIT_____________________________________

I have tried the application/json content type as suggested but it is stil happening. Tes text part of the header that is returned in the body is as follows (when using applciaiton/json encoding):

HTTP/1.1 200 OK
Date: Thu, 22 Sep 2011 08:48:28 GMT
Server: Apache/2.2.11 (Win32) mod_ssl/2.2.11 OpenSSL/0.9.8k PHP/5.3.0
X-Powered-By: PHP/5.3.0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Keep-Alive: timeout=5, max=89
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/json

How do I turn off chunked encoding for this particular script?

**EDIT 2_________________________________

I have now added a content length to my headers and the response header I get is still printed out in the body as:

HTTP/1.1 200 OK
Date: Thu, 22 Sep 2011 11:55:39 GMT
Server: Apache/2.2.11 (Win32) mod_ssl/2.2.11 OpenSSL/0.9.8k PHP/5.3.0
X-Powered-By: PHP/5.3.0
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 372797
Keep-Alive: timeout=5, max=90
Connection: Keep-Alive
Content-Type: application/json

So it looks like it is no longer being sent as chunked. However the same problem exists - the response has content then the header printed out then more content.

**The only difference now is it doesnt have the '5b03d' or '0' characters in the response.

EDIT_3___________________________

As asked for here is a summary of my php code

$dataArray = array(
    'ResultsData'=>array(
          array('test'=>'test length'),
          array('test'=>'test length'),
          array('test'=>'test length'),
          ...
));

$return = json_encode($dataArray);

header('Content-Length: '.strlen($return)); 
header('Content-type: application/json');

echo $return;

Answer 1

what you are seeing here is chunked transfer encoding - http://en.wikipedia.org/wiki/Chunked_transfer_encoding

this causes problems in combination with the content type being text/html. setting the content type to application/json should fix the problem.

Answer 2

I have a Java server that responds with json code, giving me the corruption you seem to describe. When the returned json grows big, the text is corrupted. Different browsers shows different corruptions, but the same browser very often corrupts in the same places.

In one of my tests I sendt 9492 bytes. The first 1495 were OK, the next 5204 were missing, although a Wireshark analyses showed the bytes in the tcp stream. Then the next 1495 bytes arrived safely, while the next 1298 were missing.

These numbers are an example. Other browsers will present different corruptions for the same sending. The same browsers may repeat almost the same numbers. For example is the first corruption almost always on the same byte for Chrome.

A json amount limited to say 4000 bytes always succed.

I have not found a way to stop chunkin by setting content-length in this context. Just to give you an idea of the codes credibility: The same Javaserver has served browsers with html for more than 12 years without corruption. But the Json response is sendt by the outputstream without me setting the headers.

I think the headers are composed by the apache server when I send from my own java-program.

Answer 3

I have not found an answer to this question however have isolated the problem.

In a standard php file I wrote this code:

<?php

// Make a large array of strings
for($i=0;$i<10000;$i++)
{        
  $arr[] = "testing this string becuase it is must longer than all the rest to see if we can replicate the problem. testing this string becuase it is must longer than all the rest to see if we can replicate the problem. testing this string becuase it is must longer than all the rest to see if we can replicate the problem.";
}

// Create one large string from array
$var = implode("-",$arr);

// Set HTTP headers to ensure we are not 'chunking' response
header('Content-Length: '.strlen($var)); 
header('Content-type: text/html');

// Print response
echo $var;

?>

and went to it in a browser and got the same problem.

Could someone else try this for me? (I have tried on two computers now with same result)