Is there a way to access the Google Cloud metadata service from AppEngine Standard for runtime configuration?

Question

I would like to access the Metadata Service from a App Engine Standard application. I tried doing a urlfetch to http://metadata.google.internal/computeMetadata/v1/project/attributes and got back DNS lookup failed:

    logging.info(urlfetch.fetch('http://metadata.google.internal/computeMetadata/v1/project/attributes/').content)

Is this possible? I'd like to share config between App Engine Flex and Standard code in the same project.

Answer 1

Some friendly folks on the GCP slack channel pointed me to the RuntimeConfig API for sharing configuration across multiple types of services in Google Cloud. This solves the problem of sharing configs that I was looking for.

For those curious you have to:

1. Enable the API in API Manager for your google cloud project

2. Run some gcloud commands:

   gcloud beta deployment-manager runtime-configs create foo-credentials
   gcloud beta deployment-manager runtime-configs variables set "bar-variable-name" "baz-value"  --config-name "foo-credentials"```
   

3. Add the python google-cloud-runtimeconfig library to your project (I did it via pip)

4. Add some python code to fetch the variable at runtime:

   config_client = runtimeconfig.Client()
   config = config_client.config('foo-credentials')
   bar = config.get_variable('bar-variable-name')```
   

Answer 2

No, you can't access the (GCE-specific) metadata from a GAE standard instance since it's not a GCE VM/instance. From Getting metadata (emphasis mine):

You can query the contents of the metadata server by making a request to the following root URLs from within a virtual machine instance. Use the http://metadata.google.internal/computeMetadata/v1/ URL to make requests to the metadata server.

The DNS failure you see for metadata.google.internal is a likely indicator that it's a special host DNS entry available only inside the GCE network or machine.

But in general it is possible to share files across GAE services/modules by symlinking the same file (ideally placed in the app dir) inside each of the service/module dir requiring it. See examples here: Sharing entities between App Engine modules and here: https://stackoverflow.com/a/34111170/4495081

As long as the flex service/module uses the same file(s) content(s) the same way as the standard one does, this technique should work for them as well, meaning you can share configs by sharing an appengine_config.py file, for example.