How do I run two processes in Procfile?

Question

I have a Flask app where I have embedded a Bokeh server graph and I am not being able to have them both working on Heroku. I am trying to deploy on Heorku and I can either start the Bokeh app or the Flask app from the Procfile, but not both at the same time. Consequently, either the content served with Flask will show, or the Bokeh graph.

When I deploy with the following line in Procfile, the Bokeh content shows up on the webpage, but not nothing from Flask:

web: bokeh serve --port=$PORT --host=bokehapp.herokuapp.com --host=* --address=0.0.0.0 --use-xheaders bokeh_script.py

If I deploy with the following, I only get the Flask content, not the Bokeh graph:

web: gunicorn app:app

In the second case, I am starting Bokeh inside the app.py Flask script using a subprocess:

bokeh_process = subprocess.Popen(
    ['bokeh', 'serve','--allow-websocket-origin=bokehapp.herokuapp.com','--log-level=debug','standard_way_with_curdoc.py'], stdout=subprocess.PIPE)

The Heroku logs don't show any errors.

I also tried a third alternative:

web: bokeh serve --port=$PORT --host=bokehapp.herokuapp.com --host=* --address=0.0.0.0 --use-xheaders bokeh_script.py
web: gunicorn app:app

And that shows Flask content only. It seems only second worker is being considered.

So, my question is how modify the Procfile to consider both processes? Or maybe I am approaching this wrong all together? Any clue you can give would be appreciated.

Answer 1

Each Heroku dyno gets allocated a single public network port, which you can get from the $PORT variable, pre-assigned by Heroku before launching your app. The unfortunate side effect of this is that you can have only one public web server running in a dyno.

I think the first thing that you need to do is route all requests to your application through your Flask server. For example, you can add a /bokeh/<path:path> route to your Flask app that forwards all requests to the bokeh server using requests, then sends the response back to the client. With this change, you now have a single public web server, and the bokeh server can run as a background service without public access.

So now you can deploy your Flask app to Heroku, and have it receive both its own requests and the requests for the bokeh server. The next step is to figure out where to deploy the bokeh server.

The proper way to do this is to deploy bokeh on a separate dyno. The Flask dyno will know how to forward requests to bokeh because you will have a configuration item with the bokeh server URL.

If you want to have everything hosted on a single dyno I think you can as well, though I have never tried this myself and can't confirm it is viable. The all in one configuration is less ideal and not what Heroku recommends, but according to the dyno networking documentation it appears you can privately listen on any network ports besides $PORT. Those are not exposed publicly, but the doc seems to suggest the processes running inside a dyno can communicate through private ports. So for example, you can start the Flask app on $PORT, and the bokeh server on $PORT + 1, and have Flask internally forward all the bokeh requests to $PORT + 1.