Menu
I have read this question but it doesn't exactly answer my question.
Unfortunately, it looks like things have changed in in the XHR object since I last looked at AJAX, so it is no longer possible to directly access responseText before it is finished being populated.
I have to write a page that uses AJAX (preferably jQuery, but I am open to suggestions) to retrieve CSV data via HTTP from a server I have no control over. The response data could be quite large; a megabyte of text is not uncommon.
The server is stream-friendly. Is there still any way to get access to a stream of data as it is being returned, directly from JavaScript?
I do have the option of writing some PHP code that lives in the middle and uses some sort of "Comet" tech (long-polling, EventSource, etc), but I would prefer to avoid that if possible.
In case it is relevant, assume for this question that users have the latest version of Firefox/Chrome/Opera and old browser compatibility is not an issue.
305
This is quite straightforward when outputting text or HTML. Below is an example.
(You'll run into issues if trying to output JSON however, which I'll tackle further down.)
header('Content-type: text/html; charset=utf-8');
function output($val)
{
echo $val;
flush();
ob_flush();
usleep(500000);
}
output('Begin... (counting to 10)');
for( $i = 0 ; $i < 10 ; $i++ )
{
output($i+1);
}
output('End...');
<!DOCTYPE>
<html>
<head>
<title>Flushed ajax test</title>
<meta charset="UTF-8" />
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js"></script>
</head>
<body>
<script type="text/javascript">
var last_response_len = false;
$.ajax('./flushed-ajax.php', {
xhrFields: {
onprogress: function(e)
{
var this_response, response = e.currentTarget.response;
if(last_response_len === false)
{
this_response = response;
last_response_len = response.length;
}
else
{
this_response = response.substring(last_response_len);
last_response_len = response.length;
}
console.log(this_response);
}
}
})
.done(function(data)
{
console.log('Complete response = ' + data);
})
.fail(function(data)
{
console.log('Error: ', data);
});
console.log('Request Sent');
</script>
</body>
</html>
It's not actually possible to load a single JSON object incrementally (before it's fully loaded) because until you have the complete object, the syntax will always be invalid.
But if your response has multiple JSON objects, one after another, then it's possible to load one at a time, as they come down the pipe.
So I tweaked my code above by...
Changing PHP FILE line 4 from echo $val; to echo '{"name":"'.$val.'"};'. This outputs a series of JSON objects.
Changing HTML FILE line 24 from console.log(this_response); to
this_response = JSON.parse(this_response);
console.log(this_response.name);
Note that this rudimentary code assumes that each "chunk" coming to the browser is a valid JSON object. This will not always be the case because you cannot predict how packets will arrive - you may need to split the string based on semi-colons (or come up with another separator character).
application/json
Do NOT For change your headers to application/json - I did this and it had me Googling for 3 days. When the response type is application/json, the browser waits until the response is complete, as in fully complete. The full response is then parsed to check if it is infact JSON. However our FULL response is {...};{...};{...}; which is NOT valid JSON. The jqXHR.done method assumes there was an error, because the complete response cannot be parsed as JSON.
As mentioned in the comments, you can disable this check on the client side by using:
$.ajax(..., {dataType: "text"})
Hope some people find this useful.
89
Use XMLHttpRequest.js
https://github.com/ilinsky/xmlhttprequest
http://code.google.com/p/xmlhttprequest
To use long polling with PHP:
output.php:
<?php
header('Content-type: application/octet-stream');
// Turn off output buffering
ini_set('output_buffering', 'off');
// Turn off PHP output compression
ini_set('zlib.output_compression', false);
// Implicitly flush the buffer(s)
ini_set('implicit_flush', true);
ob_implicit_flush(true);
// Clear, and turn off output buffering
while (ob_get_level() > 0) {
// Get the curent level
$level = ob_get_level();
// End the buffering
ob_end_clean();
// If the current level has not changed, abort
if (ob_get_level() == $level) break;
}
// Disable apache output buffering/compression
if (function_exists('apache_setenv')) {
apache_setenv('no-gzip', '1');
apache_setenv('dont-vary', '1');
}
// Count to 20, outputting each second
for ($i = 0;$i < 20; $i++) {
echo $i.str_repeat(' ', 2048).PHP_EOL;
flush();
sleep(1);
}
run.php:
<script src="http://code.jquery.com/jquery-1.6.4.js"></script>
<script src="https://raw.github.com/ilinsky/xmlhttprequest/master/XMLHttpRequest.js"></script>
<script>
$(function() {
var xhr = new XMLHttpRequest();
xhr.open('GET', '/longpoll/', true);
xhr.send(null);
var timer;
timer = window.setInterval(function() {
if (xhr.readyState == XMLHttpRequest.DONE) {
window.clearTimeout(timer);
$('body').append('done <br />');
}
$('body').append('state: ' + xhr.readyState + '<br />');
console.log(xhr.responseText);
$('body').append('data: ' + xhr.responseText + '<br />');
}, 1000);
});
</script>
This should output:
state: 3
data: 0
state: 3
data: 0 1
state: 3
data: 0 1 2
state: 3
data: 0 1 2 3
state: 3
data: 0 1 2 3 4
...
...
...
state: 3
data: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
state: 3
data: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
state: 3
data: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
done
state: 4
data: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
For IE you need to look into XDomainRequest
http://blogs.msdn.com/b/ieinternals/archive/2010/04/06/comet-streaming-in-internet-explorer-with-xmlhttprequest-and-xdomainrequest.aspx
http://msdn.microsoft.com/en-us/library/cc288060(VS.85).aspx
22
You're going to want to use straight up javascript for this. The reason is that you're going to want to continuously poll and not wait for the callbacks to fire. You don't need jQuery for this, it's pretty simple. They have some nice source code for this on the Ajax Patterns website.
Essentially, you'll just want to keep track of your last position in the response and periodically poll for more text past that location. The difference in your case is that you can subscribe to the complete event and stop your polling.
21
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
