Long polling PHP returns 2 results instead of one

Question

I'm trying to create a Posting System just like Facebook. So I did a little bit research about how Facebook does it, Facebook uses long polling، So I searched around on how to implement it, I implement it. And I finally finished it, I opened both Firefox and Chrome to test it out. After 2 or 3 posts it worked, but then it will duplicate the results. As you can see below:

It's the first post by the way.

And here is my network tab, During that process:

It makes 3 requests instead of one.

And finally here is my code:

init.js that contains all of my JavaScript code

function getNewPosts(timestamp) {
  var t;
  $.ajax({
    url: 'stream.php',
    data: 'timestamp=' + timestamp,
    dataType: 'JSON',
})
  .done(function(data) {
    clearInterval( t );
    // If there was results or no results
    // In both cases we start another AJAX request for long polling after 1 second
    if (data.message_content == 'results' || data.message_content == 'no-results') {
        t = setTimeout( function() {
            getNewPosts(data.timestamp);
        }, 1000);
        // If there was results we will append it to the post div
        if (data.message_content ==  'results') {
            // Loop through each post and output it to the screen
            $.each(data.posts, function(index, val) {
                $("<div class='post'>" + val.post_content + "<div class='post_datePosted'>"+ val.posted_date +"</div> <br>" + "</div>").prependTo('.posts');
            });
        }
    }
})
}

$(document).ready(function(){

    // Start the autosize function
    $('textarea').autosize();

    // Create an AJAX request to the server for the first time to get the posts
    $.ajax({
        async: false,
        url: 'stream.php?full_page_reload=1',
        type: 'GET',
        dataType: 'JSON',
    })
    .done(function(data) {
        // Assign the this variable to the server timestamp
        // that was given by the PHP script
        serverTimestamp = data.timestamp;
        $.each(data.posts, function(index, val) {
            $("<div class='post'>" + val.post_content + "<div class='post_datePosted'>"+ val.posted_date +"</div>" + "</div>").prependTo('.posts');
        });
    })
    .fail(function() {
        alert('There was an error!');
    })
    // When the form is submitted
    $('#post_form').on('submit', function(event) {
        $.ajax({
            url: 'ajax/post.php',
            type: 'POST',
            dataType: 'JSON',
            data: $('#post_form').serialize()
        })
        .done(function(data) {
            // Reset the form values
            $('#post_form')[0].reset();
        })
        .fail(function() {
            // When there was an error
            alert('An error occured');
        })
        // Prevent the default action
        event.preventDefault();
    });
    // Start the actual long polling when DOM is ready
    getNewPosts(serverTimestamp);
});

And my stream.php

<?php
header('Content-type: application/json');
// If it was a full page reload
$lastId = isset($_GET['lastId']) && !empty($_GET['lastId']) ? $_GET['lastId'] : 0;
if (isset($_GET['full_page_reload']) && $_GET['full_page_reload'] == 1) {
    $first_ajax_call = (int)$_GET['full_page_reload'];

    // Create a database connection
    $pdo = new PDO('mysql:host=localhost;dbname=test', 'akar', 'raparen');
    $sql = "SELECT * FROM `posts`";
    $stmt = $pdo->prepare($sql);
    $stmt->execute();
    $posts = $stmt->fetchAll(PDO::FETCH_ASSOC);
    // Output the timestamp since its full page reload
    echo json_encode(array(
        'fullPageReload' => 'true',
        'timestamp' => time(),
        'posts' => $posts
        ));
} else if (isset($_GET['timestamp'])) {
    // The wasted time
    $time_wasted = 0;
    // Database connection
    $pdo = new PDO('mysql:host=localhost;dbname=test', 'akar', 'raparen');
    $timestamp = $_GET['timestamp'];
    // Format the timestamp to SQL format
    $curr_time = date('Y-m-d H:i:s', $timestamp);
    $sql = "SELECT * FROM `posts` WHERE posted_date >= :curr_time";
    $stmt = $pdo->prepare($sql);
    $stmt->bindValue(':curr_time', $curr_time);
    $stmt->execute();
    // Fetch the results as an Associative array
    $posts = $stmt->fetchAll(PDO::FETCH_ASSOC);
    // If there wasn't any results
    if ( $stmt->rowCount() <= 0 ) {
        // Create the main loop
        while ($stmt->rowCount() <= 0) {
            // If there is still no results or new posts
            if ($stmt->rowCount() <= 0) {
                // If we waited 60 seconds and still no results
                if ($time_wasted >= 60) {
                    die(json_encode(array(
                        'message_type' => 'error',
                        'message_content' => 'no-results',
                        'timestamp' => time()
                        )));
                }
                // Helps the server a little bit
                sleep(1);
                $sql = "SELECT * FROM `posts` WHERE posted_date >= :curr_time";
                $stmt = $pdo->prepare($sql);
                $stmt->bindParam(':curr_time', $curr_time);
                $stmt->execute();
                $posts = $stmt->fetchAll(PDO::FETCH_ASSOC);
                // Increment the time_wasted variable by one
                $time_wasted += 1;
            }
        }
    }
    // If there was results then we output it.
    if ($stmt->rowCount() > 0) {
        die( json_encode( array(
            'message_content' => 'results',
            'timestamp' => time(),
            'posts' => $posts,
            )));
        exit();
    }
}

And here is my ajax/post.php:

<?php
if ( isset($_POST['post_content']) ) {
    $post_content = strip_tags(trim($_POST['post_content']));
    if ( empty($post_content) ) {

        /* If the user doesn't enter anything */
        echo json_encode(array(
            'message_type' => 'error',
            'message_content' => 'It seems like your post is empty'
            ));
    } else {
        $pdo = new PDO('mysql:host=localhost;dbname=test', 'akar', 'raparen');
        $sql = "INSERT INTO `posts` (`post_id`, `post_content`, `posted_date`) VALUES (NULL, :post_content, NOW());";
        $stmt = $pdo->prepare($sql);
        $stmt->bindValue(':post_content', $post_content);
        $stmt->execute();
        echo json_encode(array(
            'message_type' => 'message',
            'message_content' => 'Your post has been posted successfully.'
            ));
    }
}

If you don't understand it just ask me. I know it's dirty code and I repeated myself a lot. I did that for testing, so it doesn't really matter.

Thanks!

Answer 1

Frankly I don't see why you botyher with this kind of optimization, unless you plan to handle thousands of messages. You could as well fetch the whole lot each time the page gets refreshed.

Hammering the server with a request from each client every second will generate a huge lot of traffic anyway, so optimizations should start with defining a more reasonable polling period or a smarter, adaptative refresh mechanism, IMHO.

Now if you really want to go for it, you will have to do a proper synchronization. If you mess up the timestamps, you can skip a message that was added by someone else just as another client triggered an auto-refresh, or get the said message twice.

All your timeout handling is simply unnecessary. A server request through Ajax will produce an error event if something goes wrong, which will mean either the connection or your server went down or your PHP-side code threw some tantrum and needs a fix.

For an idea of the application structure:

• an update request will pass a timestamp to PHP, asking to retrieve all posts newer than the said timestamp. The initial value will be 1/1/1970 or whatever, so that the initial fetch retrieves all existing posts. A new timestamp will be included in the response, so that incremental requests will skip already fetched data.
• Javascript will generate such requests periodically (I would rather set the period to 30 seconds or so, to avoid excessive server load - assuming your average user can handle the frustration of waiting that long for the next batch of pseudo-tweets)
• submitting a new post will simply add it to the DB, but since all is done server-side, you won't need to bother with race conditions.

All your "time_wasted" and "cur_time" code should go to the bin. The only time reference needed is the date of the last read request from this particular client.
All you need server-side (in your "stream" PHP file) is a DB request to fetch posts newer than the client-provided timestamp, that will return a (possibly empty) list of posts and the updated value of the same timestamp.

Frankly, instead of these potentially confusing timestamps, you could as well use the unique identifier of the last post fetched (using 0 or any conventional value for the initial request).

Answer 2

You set a timeout like:

setTimeout()

but you are using

clearInterval()

to clear it, use clearTimeout() instead.

Answer 3

I know this does not answer your question exactly, but what you're doing is not gonna work anyway - long polling with PHP will crash your server when there'll be at least some more users. You use sleep, so PHP process is "hanging". PHP worker count (both for Apache, nginx or any server with PHP) is limited. As soon as you'll reach that count, new connections will be rejected. PHP is intended to give responses quickly.

For this type of solution I would suggest to use some intermediate software that's designed for it. For example, take a look at Socket.IO.

It's written in Javascript and is for both client side (JS library) and server side (Node.js). Your Node.js server can take events as they happen from PHP using REST API, queues (like ZeroMQ, RabbitMQ etc) or any other transport, like socket.IO client itself. This way you don't poll your database in PHP, you just pass that new post was added to the Node.js server, which passes this information to your client-side JS code.

$pdo->prepare('INSERT INTO ..')->execute();
$dispatcher->dispatch('new_post', new PostEvent(array('id' => 123, 'text' => 'Post text')));

Long-polling is only one of supported Socket.IO protocols, and is by far not the most efficient one.

If you want to avoid Node.js, you can try ReactPHP with Ratchet, using WebSockets on client side. This works as single php process (ran from command line), thus not apache-way.