GAE Go - "This request caused a new process to be started for your application..."

Question

I've encountered this problem for a second time now, and I'm wondering if there is any solution to this. I'm running an application on Google App Engine that relies on frequent communication with a website through HTTP JSON RPC. It appears that GAE has a tendency to randomly display a message like this in the logs:

"This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application."

And reset all variables stored in RAM without warning. The same process happens over and over no matter how many times I set the variables again or upload newer code to GAE, although incrementing the app version number seems to solve the problem.

How can I get more information on this behaviour, how to avoid it and prevent data loss of my Golang applications on Google App Engine?

EDIT:

The variables stored in RAM are small classes of strings, bytes, bools and pointers. Nothing too complicated or big.

Google App Engine seems to "start a new process" in matter of seconds of heavier use, which shouldn't be long enough time for the application to be shut down for not being used. The timespan between application being uploaded to GAE, having its variable set and a new process being created is less than a minute.

Answer 1

Do you realize that GAE is a cloud hosting solution that automatically manages instances based on the load? This is it's main feature and reason people are using it.

When load increases, GAE creates a new instance, which , of course, has all RAM variables empty.

The solution is not to expect variables to be available or store them to permanent storage at the end of request (session, memcache, datastore) and load them if not present at the beginnig of request.

Answer 2

You can read about GAE instances in their documentation here, check out the performance section:

http://code.google.com/appengine/kb/java.html

In your case of having small data available, if its static then you can load it into memory on startup of a new instance. If it's dynamic data, you should be saving it to the database using their api.

My recommendation for keeping a GAE instance alive, either pay for the Always-On service or follow my recommendations for using a cron here:

http://rwyland.blogspot.com/2012/02/keeping-google-app-engine-gae-instances.html

I use what I call a "prime schedule" of a 3, 7, 11 minute cron job.

Answer 3

You should consider using Backends if you want long running instances with resident memory.