Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Event Source -> Server returns event stream in bulk rather then returning in chunk

I have a php script that import large data from csv files with validations.
For that I need to show progress to the user. I have used Event Streaming for that.
When I echo something, I want it to be transferred to client one by one instead of server sent whole output in bulk.
I had already played around with ob_start(), ob_implicit_flush() & ob_flush(), but they didn't work.
My script is working perfect on another server. Below server configurations are given:

Server configuration on which the code is not responding as desired, i.e.

OS: Linux
PHP Version 5.4.36-0+deb7u3
Server API: CGI/FastCGI 
Memory_limit: 128M
output_buffering: no value

As I have said, the code is working properly on another server which has the almost same configuration, i.e.

OS: Linux
PHP Version 5.4.37
Server API: CGI/FastCGI 
Memory_limit: 256MB
output_buffering: no value

Below is my sample code for sending event:

<?php
header("Content-Type: text/event-stream");
header("Cache-Control: no-cache");
header("Access-Control-Allow-Origin: *");

$lastEventId = floatval(isset($_SERVER["HTTP_LAST_EVENT_ID"]) ? $_SERVER["HTTP_LAST_EVENT_ID"] : 0);
if ($lastEventId == 0) {
    $lastEventId = floatval(isset($_GET["lastEventId"]) ? $_GET["lastEventId"] : 0);
}

echo ":" . str_repeat(" ", 2048) . "\n"; // 2 kB padding for IE
echo "retry: 2000\n";

// event-stream
$i = $lastEventId;

while ($i <= 100) {
    if($i==100){
        echo "data: stop\n";
        ob_flush();
        flush();
        break;
    } else {
        echo "id: " . $i . "\n";
        echo "data: " . $i . ";\n\n";
        ob_flush();
        flush();
        sleep(1);
    }
    $i++;
}
?>

Below is my client page on which I need response:

<!DOCTYPE html>
<html>

<head>
    <meta charset="utf-8" />
    <title>EventSource example</title>
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <script src="../jquery/eventsource.js"></script>
    <script>
        var es = new EventSource("events.php");
        var listener = function(event) {
            console.log(event.data);
            var type = event.type;
            if (event.data == 'stop') {
                es.close();
            } else {
                var div = document.createElement("div");
                div.appendChild(document.createTextNode(type + ": " + (type === "message" ? event.data : es.url)));
                document.body.appendChild(div);
            }
        };
        var errlistener = function(event) {
            es.close();
        }
        es.addEventListener("open", listener);
        es.addEventListener("message", listener);
        es.addEventListener("error", errlistener);
    </script>
</head>

<body>
</body>

</html>
like image 762
ba1ar Avatar asked Feb 19 '15 13:02

ba1ar


People also ask

How does event Source work?

An EventSource instance opens a persistent connection to an HTTP server, which sends events in text/event-stream format. The connection remains open until closed by calling EventSource. close() . Once the connection is opened, incoming messages from the server are delivered to your code in the form of events.

What is HTTP event stream?

Event stream format. The event stream is a simple stream of text data which must be encoded using UTF-8. Messages in the event stream are separated by a pair of newline characters.

How do I send events from server to client?

The server-sent events streaming can be started by the client's GET request to Server. Accept: text/event-stream indicates the client waiting for event stream from the server, Cache-Control: no-cache indicates that disabling the caching and Connection: keep-alive indicates the persistent connection.

Which method of EventSource object will be invoked when an error occurs?

The onerror event occurs when an error occurs with the event source. An error usually occurs when a connection is disrupted. If this happens, the EventSource object will automatically attempt to reconnect to the server.


1 Answers

Your best method to return chucked data to the browser is to use Web Sockets get the client to open a socket to your file reader then you can chunk the data to the browser without a problem.

Then once it has finished you can close the socket.

a good tutorial for web sockets http://www.phpbuilder.com/articles/application-architecture/optimization/creating-real-time-applications-with-php-and-websockets.html

with this method you can then if you wanted implement verification so the server is not just sending chunks it's sends the chunks on request by javascript

So your Client could say i need chunk 5 and your server implement something like

$requestedChunk = 5; // this would be set by the javascript sending the request
$chunkSize = 256; // this would be your chunk size;

$readPossition = $requestedChunk * $chunkSize;

Link no longer works so here is one built on Ratchet: https://blog.samuelattard.com/the-tutorial-for-php-websockets-that-i-wish-had-existed/

like image 87
Barkermn01 Avatar answered Oct 26 '22 10:10

Barkermn01