Loading global data for server using Flask and gunicorn

Question

I have implemented simple API using the Flask framework and now I am trying to deploy it to gunicorn server.

My server script looks as follows:

app = Flask(__name__)

class Server(object):
   def __init__(self, data):
       self.data = data

@app.route("/api_method", methods=['GET', 'POST'])
def api_method():
     return server.data

if __name__ == '__main__':
     with smart_open(sys.argv[1]) as f:
         server = Server(f.read())
     app.run()

When I run this from console as "Flask app" than everything is ok, but when I try to run this under gunicorn then it does not see the server. I was able to fixe it only by moving server creation up, but I had to hardcode the path.

Is there some way how to load something like my Server class at the startup of the gunicorn server and then read it within API methods?

Answer 1

Here are my recommendations on how to do this.

1. I wouldn't rely on the global server variable if it needs to be instantiated this way. Since it's instantiated after the app's creation it would be better to pass it explicitly into the application somehow. This way you can access it from other modules (blueprints, pluggable views, etc.) For example, you can use Flask's configuration mechanism:

app.config['server'] = Server(f.read())

#...then in your view functions...

return app.config['server'].data

Or through flask's globals object, g:

@app.before_request
def add_server_to_globals():
    with open(sys.argv[1]) as f:
        g.server = Server(f.read())

#...then in your view functions...
return g.server.data

Or by subclassing Flask and passing it in through the initializer.

2. Now that you're using gunicorn, you cannot rely on the if __name__ == '__main__' block, as this is only executed when you run the file, not when it's imported. Gunicorn works by importing the application object (see how APP_MODULE is explained in the docs) and handing it off to gunicorn's WSGI server, so your code for instantiating Server will not run.

The other new requirement from unicorn is that you cannot rely on sys.argv, since this will be gunicorn's sys.argv, which is different from whatever argument you are currently passing in when running the file directly.

To replace the if __name__ == '__main__' block you can use before_request as I did above to run some code before the application handles its first request. I recommend using an environment variable or a config file to pass in your argument to the Server class. For example:

@app.before_request
def add_server_to_globals():
    with open(os.environ['SERVER_FILE']) as f:
        g.server = Server(f.read())

Then run your application with the environment variable set:

$ SERVER_FILE=blah.txt gunicorn app:app
[2015-02-24 10:10:20 -0800] [3217] [INFO] Starting gunicorn 19.2.1
...

3. You can also take a look at the "application factory" pattern described in the flask docs to replace the if __name__ == '__main__' block. But in this case it's not quite necessary. The use for an application factory would be if you want to instantiate the Server object only once (for performance reasons).

Here's some fully working code based on your example.

Here's another example using an application factory.

Answer 2

Simply :)

with smart_open(sys.argv[2]) as f:
    server = Server(f.read())
if __name__ == '__main__':
    app.run()

Logic is already give you by Steven Kryskalla in his answer.

But ensure that you keep app.run() inside if __name__=="__main__" since you do not want that to be run when gunicorn imports server

EDIT: There has to be actually sys.argv[2] instead, because in the sys.argv[1] is the path to the server app.