Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Monitoring a real-time data stream with a flask web-app

This is based on the answer posted at https://stackoverflow.com/a/13388915/819544

I would like to monitor a stream of data and push it to a front-end similar to the answer above, but where the stream starts generating/monitoring data as soon as the app kicks off, and the client always sees the current state of the data steam (which keeps running whether or not they are requesting data from the server).

I'm pretty sure I need to separate the data stream from the front-end via threading, but I'm not very practiced with threaded/asynchronous programming and believe I'm doing it wrong. Maybe instead of threading I need to be using multiprocessing? Here's roughly what I'm trying to do (modified from the answer linked above):

app.py

#!/usr/bin/env python
from __future__ import division
import itertools
import time
from flask import Flask, Response, redirect, request, url_for
from random import gauss
import threading

app = Flask(__name__)

# Generate streaming data and calculate statistics from it
class MyStreamMonitor(object):
    def __init__(self):
        self.sum   = 0
        self.count = 0
    @property
    def mu(self):
        try:
            outv = self.sum/self.count
        except:
            outv = 0
        return outv
    def generate_values(self):
        while True:
            time.sleep(.1)  # an artificial delay
            yield gauss(0,1)
    def monitor(self, report_interval=1):
        print "Starting data stream..."
        for x in self.generate_values():
            self.sum   += x
            self.count += 1 

stream = MyStreamMonitor()

@app.route('/')
def index():
    if request.headers.get('accept') == 'text/event-stream':
        def events():
            while True:
                yield "data: %s %d\n\n" % (stream.count, stream.mu)
                time.sleep(.01) # artificial delay. would rather push whenever values are updated. 
        return Response(events(), content_type='text/event-stream')
    return redirect(url_for('static', filename='index.html'))

if __name__ == "__main__":
    # Data monitor should start as soon as the app is started.
    t = threading.Thread(target=stream.monitor() )
    t.start()
    print "Starting webapp..." # we never get to this point.
    app.run(host='localhost', port=23423)

static/index.html

<!doctype html>
<title>Server Send Events Demo</title>
<style>
  #data {
    text-align: center;
  }
</style>
<script src="http://code.jquery.com/jquery-latest.js"></script>
<script>
if (!!window.EventSource) {
  var source = new EventSource('/');
  source.onmessage = function(e) {
    $("#data").text(e.data);
  }
}
</script>
<div id="data">nothing received yet</div>

This code doesn't work. The "Starting webapp..." message never prints, nor do the normal flask messages, and visiting the served URL confirms the app isn't running.

How do I get the data monitor to run in the background in such a way that flask can access the values it's seeing and push the current state up to the client (better even: as long as the client is listening, push the current state when the relevant values change)?

like image 731
David Marx Avatar asked Sep 27 '22 03:09

David Marx


People also ask

How do you stream live video frames from client to flask server and back to the client?

SocketIo connection will get establish and video stream captured using webcam will be send to server frames by frames. These frames will be then processed at the backend and emit back to the client. Processed frames coming form the server can be shown in img tag.

How do you expose the flask app on the Internet?

Today, there are two ways to expose your flask application to the internet. Deploy the web application in your office server which has a public IP address and domain name. Deploy the web application in the cloud such as AWS, MS Azure, GCP or web hosting companies like GoDaddy, SiteGround, A2Hosting etc.

Is flask good for big projects?

Flask isn't suitable for complex systems If you're trying to build a complex web app with lots of dynamic pages, then Flask isn't the right framework to use. Over time, the maintenance costs and efforts will be much higher compared to a framework like Django, which is better suited for large projects.


1 Answers

I just needed to change this line

t = threading.Thread(target=stream.monitor())

to this:

t = threading.Thread(target=stream.monitor)
like image 116
David Marx Avatar answered Oct 20 '22 08:10

David Marx