Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

PHP/JS Progress Bar

I'm trying to make a page which will generate a result-set from a complex database query & php parsing... but that's mainly beside the point... The main point is that this takes a minute or two to complete, and I'm hoping to display a progress bar rather then a generic gif animation "loading..." picture.

A breakdown would be...

  • User opens Page A.
  • Page A requests data from Page B (Most likely AJAX).
  • Page B processes the 100000+ or so entries in the database and parses them.
  • Page A shows a progress bar which shows roughly how far through the process is
  • Page B returns the result set.
  • Page A displays the result set.

I know how to return data to the ajax query, but my issue is I don't know how to continuously return data to show the status of the process (Eg. % of rows scanned).

I've looked into EventSource / Server-Sent-Events, which shows promise, I'm just not too sure how to get it working properly, or if there is a better way to do it.

I've tried making a quick little mock-up page, using just EventSource works fine, but when I split it up into an eventSource call (page which monitors a session variable for change), and an ajax request (the actual data sending/return) it falls apart.

I'm probably missing something obvious, or doing something stupidly wrong, but this is most of what I have anyway... Any help, suggestions, tips, or even suggestions of completely other ways to do it would be awesome :)

User page:

<!DOCTYPE html>
<html>
<head>
    <title>Dynamic Progress Bar Example</title>
    <script src="script.js"></script>
</head>
<body>
    <input type="button" value="Submit" onclick="connect()" />
    <progress id='progressor' value="0" max='100' style=""></progress>
</body>
</html>

Javascript

 var es;

   function connect() {
       startListener();
       $.ajax({
           url: "server.php",
           success: function() {
               alert("Success");
           },
           error: function() {
               alert("Error");
           }
       });
   }

   function startListener() {
       es = new EventSource('monitor.php');

       //a message is received
       es.addEventListener('message', function(e) {
           var result = JSON.parse(e.data);

           if (e.lastEventId == 'CLOSE') {
               alert("Finished!");
               es.close();
           } else {
               var pBar = document.getElementById('progressor');
               pBar.value = result;
           }
       });

       es.addEventListener('error', function(e) {
           alert('Error occurred');
           es.close();
       });
   }

   function stopListener() {
       es.close();
       alert('Interrupted');
   }

   function addLog(message) {
       var r = document.getElementById('results');
       r.innerHTML += message + '<br>';
       r.scrollTop = r.scrollHeight;
   }

Monitor PHP

<?php
SESSION_START();
header('Content-Type: text/event-stream');
// recommended to prevent caching of event data.
header('Cache-Control: no-cache'); 

function send_message($id, $data) {
    $d = $data;
    if (!is_array($d)){
        $d = array($d);
    }

    echo "id: $id" . PHP_EOL;
    echo "data: " . json_encode($d) . PHP_EOL;
    echo PHP_EOL;

    ob_flush();
    flush();
}


$run = true;
$time = time();
$last = -10;

while($run){
    // Timeout kill checks
    if (time()-$time > 360){
        file_put_contents("test.txt", "DEBUG: Timeout Kill", FILE_APPEND);
        $run = false;
    }

    // Only update if it's changed
    if ($last != $_SESSION['progress']['percent']){
        file_put_contents("test.txt", "DEBUG: Changed", FILE_APPEND);
        $p = $_SESSION['progress']['percent'];
        send_message(1, $p); 
        $last = $p;
    }

    sleep(2);
}
?>

EDIT: I've tried a different approach, where:

  • Page A AJAX calls page B, which runs the request, and saves the progress to a SESSION variable
  • Page A AJAX calls page C every 2 seconds, which simply returns the value of the session variable. This loop is terminated when it reaches 100

However, this is not quite working either. It seems that the two AJAX requests, or the two scripts server-side are not running simultaneously.

Looking at debug output: Both AJAX calls are executed at about the same time, but then the page B script runs to completion by itself, and -then- the page C script runs. Is this some limitation of PHP I'm missing???

more code!

Server (Page B) PHP

<?PHP
    SESSION_START();

    file_put_contents("log.log", "Job Started\n", FILE_APPEND);

    $job = isset($_POST['job']) ? $_POST['job'] : 'err_unknown';
    $_SESSION['progress']['job'] = $job;
    $_SESSION['progress']['percent'] = 0;

    $max = 10;
    for ($i=0; $i<=$max;$i++){
        $_SESSION['progress']['percent'] = floor(($i/$max)*100);
        file_put_contents("log.log", "Progress now at " . floor(($i/$max)*100) . "\n", FILE_APPEND);
        sleep(2);
    }

    file_put_contents("log.log", "Job Finished", FILE_APPEND);
    echo json_encode("Success. We are done.");
?>

Progress (Page C) PHP

<?php
    SESSION_START();
    file_put_contents("log.log", "PR: Request Made", FILE_APPEND);

    if (isset($_SESSION['progress'])){
        echo json_encode(array("job"=>$_SESSION['progress']['job'],"progress"=>$_SESSION['progress']['percent']));
    } else {
        echo json_encode(array("job"=>"","progress"=>"error"));
    }
?>

Index (Page A) JS/HTML

<!DOCTYPE html>
<html>
<head>
        <title>Progress Bar Test</title>
</head>
<body>
        <input type="button" value="Start Process" onclick="start('test', 'pg');"/><br />
        <progress id="pg" max="100" value="0"/>

        <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
        <script type="text/javascript">
            var progress = 0;
            var job = "";

            function start(jobName, barName){
                startProgress(jobName, barName);
                getData(jobName);
            }

            function getData(jobName){
                console.log("Process Started");
                $.ajax({
                    url: "server.php",
                    data: {job: jobName},
                    method: "POST",
                    cache: false,
                    dataType: "JSON",
                    timeout: 300,
                    success: function(data){
                        console.log("SUCCESS: " + data)
                        alert(data);
                    },
                    error: function(xhr,status,err){
                        console.log("ERROR: " + err);
                        alert("ERROR");
                    }
                });
            }

            function startProgress(jobName, barName){
                console.log("PG Process Started");
                progressLoop(jobName, barName);
            }

            function progressLoop(jobName, barName){
                console.log("Progress Called");
                $.ajax({
                    url: "progress.php",
                    cache: false,
                    dataType: "JSON",
                    success: function(data){
                        console.log("pSUCCESS: " . data);
                        document.getElementById(barName).value = data.progress;
                        if (data.progress < 100 && !isNaN(data.progress)){
                            setTimeout(progressLoop(jobName, barName), (1000*2));
                        }
                    },
                    error: function(xhr,status,err){
                        console.log("pERROR: " + err);
                        alert("PROGRESS ERROR");
                    }
                });
            }
        </script>
</body>
</html>

Debug: log.log output

PR: Request Made
Job Started
Progress now at 0
Progress now at 10
Progress now at 20
Progress now at 30
Progress now at 40
Progress now at 50
Progress now at 60
Progress now at 70
Progress now at 80
Progress now at 90
Progress now at 100
Job Finished
PR: Request Made
like image 930
Kieran Avatar asked May 13 '15 12:05

Kieran


1 Answers

In similar cases, I usually do it this way:

  • Client sends AJAX request to Page B. Important: On success, client sends the same request again.
  • On the initial request, Page B says: OK, THERE ARE 54555 RECORDS.. I use this count to initiate the progress bar.
  • On each of next requests, Page B returns a chunk of data. Client counts the size of chunk and updates progress bar. Also it collects chunks in one list.
  • On last request, when all data is sent, Page B says: THAT'S ALL and client renders the data.

I think, you've gotten the idea.

NOTE: you can request all chunks in parallel, but it is a complex way. Server (Page B) should also return a fixed chunksize in the initial response, then client sends TOTAL_COUNT / CHUNK_SIZE requests concurrently and combines the responses till the last request is completed. So it is much faster. You can use https://github.com/caolan/async in this case to do the code much more readable.

like image 184
Kirill Rogovoy Avatar answered Oct 20 '22 04:10

Kirill Rogovoy