How do I implement server affinity or sticky sessions on App Engine?

Question

My application wishes to have:

1. Automatic scalability

   • I want App Engine to spin up new instances of my app when traffic increases
   • When instances become idle, I want App Engine to shut them down

2. Client/server affinity

   • After an initial client->server HTTP request, I want clients to be able to connect to the same appserver, so that the appserver can maintain a bunch of client state
   • State may be updated frequently (in order to support real-time interactions), so memcache+datastore based persistence is not desirable.
   • The server may need to make decisions based on the state of multiple clients, e.g. real time multi-player game

How can I accomplish this?

Answer 1

You can achieve these goals using App Engine backends (long-running, configurable, addressable, persistent servers):

• https://developers.google.com/appengine/docs/python/backends/
• https://developers.google.com/appengine/docs/java/backends/

Python implementation

1. Configure a backend to be both public and dynamic

   # backends.yaml
   
   backends:
   - name: foo
     instances: 20
     options: public, dynamic
   

2. In addition to deploying your app in usual way:

   appcfg.py update .
   

   remember to deploy you backend:

   appcfg.py backends . update
   

3. For the initial connection, have your client use the non-instance specific backend hostname, e.g.:

   foo.your_app_id.appspot.com
   

   App Engine will route your request to available backend instance, after optionally starting a new backend instance.

4. In the request handling code on the server, use the backends API to determine which instance is handling the request and return to the client a instance specific URL.

   from google.appengine.api import backends
   
   import webapp2
   
   class GetPersistentUrlHandler(webapp2.RequestHandler):
   
     def get(self):
       """Return the current backend's instance-specific URL."""
   
       my_url = backends.get_url(instance=backends.get_instance())
       self.response.write(my_url)
   
   app = webapp2.WSGIApplication([
     ('/get_peristent_url', GetPersistentUrlHandler),
   ], debug=True)
   

5. Client makes subsequent connections to the instance specific backend URL:

   http://3.foo.your_app_id.appspot.com
   

   Note: when using https be sure to replace subdomain dots with -dot- in order to avoid SSL certificate issues.

   https://3-dot-foo.your_app_id.appspot.com
   

Limitations

1. Backends do not live forever and may be shutdown unexpectedly and without notice
2. The number of backends your application can have is currently limited

Answer 2

To complement Adam's good answer: you don't need server affinity in GAE, because data stored in the HTTP session is not held in memory, but in the persistent datastore. So any server will find what any other server previously stored in the session. See the documentation:

App Engine includes an implementation of sessions, using the servlet session interface. The implementation stores session data in the App Engine datastore for persistence, and also uses memcache for speed. As with most other servlet containers, the session attributes that are set with session.setAttribute() during the request are persisted at the end of the request.

This feature is off by default. To turn it on, add the following to appengine-web.xml:

<sessions-enabled>true</sessions-enabled>

The implementation creates datastore entities of the kind _ah_SESSION, and memcache entries using keys with a prefix of _ahs.

Note: Because App Engine stores session data in the datastore and memcache, all values stored in the session must implement the java.io.Serializable interface.

It's possible to reduce request latency by configuring your application to asynchronously write HTTP session data to the datastore: