Flask sse-stream not terminated after firefox disconnects

Question

I am trying to create a Flask server that streams data to the client using sse. The piece of test-code below seems to do the trick, but I stumbled upon a problem related to handling client disconnects.

When using Firefox as the client (versions 28 or 29), data starts streaming as expected. However, when I reload the page, a new stream is opened (as expected) but the old stream still remains. The eventgen() thread handling the stream is never terminated. On other clients (I tried IE using Yaffle’s Polyfill EventSource implementation as well as Chrome), reloading or closing the page results in a client disconnect, which results in a server-side socket error 10053 (client disconnected from the host). This terminates the loop and only keeps the active streams alive, which is the expected behavior.

Using Process Explorer, I noticed that the TCP connection on the client (Firefox) side hangs in the state FIN_WAIT2, while the connection at the server side hangs in the state CLOSE_WAIT. The strange thing is that on 1 out of the 3 machines (all Win 7 x64) running Firefox I tested this on, the disconnects were handled correctly. Running on Python 2.6.5 and 2.7.6 produced the same results.

I also tried replacing the built-in Flask server with the greenlet-based gevent WSGIserver, but this results in exactly the same behavior. Furthermore, some form of threading/eventlets should be used since otherwise running the eventgen() loop blocks the server.

The test code below serves the page defined in make_html() when browsing to localhost:5000 and opens a stream to /stream. The stream shows massages of the form {"content": 0.5556278827744346, "local_id": 4, "msg": 6}, where local_id is the id of the stream opened and msg is the number of the current message in this stream.

import time, random
import flask
from flask import Flask, json

def make_html():
    return """
        <script src="http://code.jquery.com/jquery-latest.min.js" type="text/javascript"></script>
        <script type=text/javascript>
            var source = new EventSource('/stream');
            source.onmessage = function (event) {
                var data = event.data;                
                var logdiv = $('#log');
                logdiv.empty();
                logdiv.append('<div class="event">' + data + '</div>');
            };
        </script>
        <h1>Log</h1>
        <div id=log>Log ...</div>
        <hr />
    """

# ---- Flask app ----

app = Flask(__name__)

@app.route('/')
def index():
    return make_html()

counter = 0
def eventgen():
    global counter
    counter += 1    
    local_id = counter
    msg_count = 0
    while True:
        msg_count += 1
        data = {'msg': msg_count, 'content': random.random(), 'local_id': local_id}
        data = json.dumps(data)
        yield 'data: ' + data + '\n\n'
        print local_id, ':', data
        time.sleep(0.5)

@app.route('/stream')
def eventstream():
    return flask.Response(eventgen(), mimetype="text/event-stream")

if __name__ == '__main__':    
    app.run(threaded=True)

Answer 1

I seem to have found the source of this issue. The problem seems to be with the AVG surf-shield link scanner and firefox. Disabling surf-shield seems to solve the problem. The PC on which it already worked was running Avast instead of AVG. I'm guessing this is a bug in AVG, which should probably be fixed.