Running GAE Development Server of Google Compute Engine Instance <phew>

Question

I'm trying to run the local dev server (java) for Google AppEngine on a Google compute instance. (we're using compute engine instances as test servers).

When trying to start the dev server using appcfg.sh we notice that 90% of the time, the server doesn't get started and hangs for 10minutes before finnaly starting.

I know that the server hasn't started because this line is never printed to the console when it hangs:

Server default is running at http://localhost:8080/

Has anyone seen anything like this?

Answer 1

In a nutshell:

-The App Engine java SDK uses jetty as the servlet container for the development appserver

-Jetty relies on java.security.SecureRandom

-SecureRandom consumes entropy from /dev/random by default

-/dev/random will block when insufficient entropy is available for a read

The GCE instance, when lightly used (for example, solely as a test appengine server), does not generate entropy quickly. Thus, repeated startups of the java appengine server consume entropy from /dev/random more rapidly than it is replenished, causing the blocking behavior on startup that you observed as the hangs on startup.

You can confirm that the hang is due to the SecureRandom issue by increasing the logging levels of the dev appserver. You should see a message similar to "init SecureRandom" and then the blocking behavior.

Some possible ways to address this:

1) Adding the following to the dev_appserver.sh invocation will cause SecureRandom to consume the /dev/urandom entropy source rather than /dev/random:

--jvm_flag="-Djava.security.egd=file:/dev/./urandom"

2) Having a GCE instance that's more heavily utilized should cause entropy data to be collected more rapidly, which will in turn make /dev/random less susceptible to blocking on subsequent restarts of the development appserver.