Loading Data Asynchronously in Python/Flask for d3.js

Question

I have a python/flask application. In it I have a page which queries the database a lot in order to display the data on the page. However, if I include this logic in the view.py where the @app.route() is, then loading the page takes too long. I'm then displaying this data with d3.

My current setup is that I have a separate route from the view which calculates the data (let's say "/data" is the path). When you go to that path, it returns the data in a json format and is loaded by

d3.json("/data", callback)

This works fine. My issue is that there are a lot of different queries I need to complete, and having a different path for each dataset is cluttering up my app and doesn't seem to be following the DRY principle. Is there a better way to do this? I'm also having difficulties passing variables to the javascript that runs the d3 from python.

Answer 1

What you really need here is an API for accessing the data. If you're using flask already then Flask-Restful is a great option. You can set up a route like this:

from flask import Flask
from flask_restful import Resource, Api, reqparse

app = Flask(__name__)
api = Api(app)

class Data(Resource):
    def __init__(self):
        self.parser = reqparse.RequestParser()
        self.parser.add_argument('name', type=str, required = True)
        self.parser.add_argument('whatever', type=str, required = True)
    def post(self):
        args = self.parser.parse_args()
        #query your database here and do your stuff
        print(args['name'])
        print(args['whatever'])
        return {'data': 'your data here',
                'other_information': 'more_stuff' }
api.add_resource(Data, '/data')

if __name__ == '__main__':
    app.run(debug=True)

and then you'll be able to pass variables that specify what data is needed to using either JSON, query strings, or a form. Flask-Restful is pretty smart and will figure out the method that you're using to send it automatically. Then you can return a dictionary which will be encoded using JSON. This should address your second issue of passing multiple variables back to the javascript.

An example of the corresponding javascript is here:

var xhr = new XMLHttpRequest();
xhr.open("POST", "http://127.0.0.1:5000/data");
xhr.setRequestHeader('Content-Type', 'application/json;charset=UTF-8');
xhr.onload = function() {
    var dict = JSON.parse(xhr.responseText);
    console.log(dict['data']);
    console.log(dict['other_information']);
};

var request = JSON.stringify({'name':'my_name', 'whatever':'stuff'});
xhr.send(request);