Reading Java resource files in Google AppEngine

Question

I have a Google AppEngine Java project set up that calls BigQuery to display the results of a query job. The example code and instructions I used can be found here. The app runs and retrieves the query from my development machine, however when I upload to AppEngine to test on appspot.com it appears the client_secrets.json file (different file for dev and appspot.com because of JavaScript URL authorization) cannot be loaded using the following snippet:

  static GoogleClientSecrets getClientCredential() throws IOException {
if (clientSecrets == null) {

    clientSecrets = GoogleClientSecrets.load(JSON_FACTORY,
            new InputStreamReader(new FileInputStream(
                new File("WEB-INF/client_secrets.json"))));

  Preconditions.checkArgument(!clientSecrets.getDetails().getClientId().startsWith("Enter ")
      && !clientSecrets.getDetails().getClientSecret().startsWith("Enter "),
      "Enter Client ID and Secret from https://code.google.com/apis/console/?api=bigquery "
      + "into bigquery-appengine-sample/src/main/resources/client_secrets.json");
}
return clientSecrets;

}

Here is my appengine-web.xml resource file definition:

  <resource-files>
    <include path="**" />
  </resource-files>

The client_secrets.json file is packaged properly in my war file. The error I get is:

> /
Uncaught exception from servlet
java.io.FileNotFoundException: /base/data/home/apps/s~tactile-reason-849/1.385872137632330782/WEB-INF/client_secrets.json (No such file or directory)
    at java.io.FileInputStream.open(Native Method)
    at java.io.FileInputStream.<init>(FileInputStream.java:171)
    at com.google.api.client.sample.bigquery.appengine.dashboard.ServiceUtils.getClientCredential(ServiceUtils.java:71)
    at com.google.api.client.sample.bigquery.appengine.dashboard.ServiceUtils.newFlow(ServiceUtils.java:103)
    at com.google.api.client.sample.bigquery.appengine.dashboard.MainServlet.initializeFlow(MainServlet.java:125)
    at com.google.api.client.extensions.servlet.auth.oauth2.AbstractAuthorizationCodeServlet.service(AbstractAuthorizationCodeServlet.java:124)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
    at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:257)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
    at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:146)
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:482)
    at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:437)
    at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:444)
    at com.google.tracing.CurrentContext.runInContext(CurrentContext.java:230)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:308)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:300)
    at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:441)
    at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:234)
    at java.lang.Thread.run(Thread.java:745)

I can try using ServletContext.getResourceAsStream() instead, however the class which connects to BigQuery in the example app is not a Servlet class, so this will require some reorganization of the code to implement properly, and Google's instructions seem to indicate my code should work. Any help is appreciated, thanks!

Answer 1

As a workaround for AppEngine's bug I created an object containing the Firebase authentication data that I build from the FirebaseCredentialsHelper class. The returned object uses the String values generated from the Firebase console.

Then I converted my authentication object returned by the FirebaseCredentialsHelper class into json using the Gson library, and then into a byte input stream.

FirebaseApp.initializeApp(FirebaseOptions.Builder()
.setCredentials(GoogleCredentials.fromStream(Gson().toJson(FirebaseCredentialsHelper.get()).byteInputStream()))
.build()) 

Note: I kept the Firebase Authentication objects out of version control since they contain private information.

Answer 2

Resource files are not on the filesystem, so new File("/WEB-INF/client_secrets.json") ordinary wouldn't work. Instead, you usually need to load it as a resource:

InputStream resourceStream = Thread.currentThread().getContextClassLoader()
    .getResourceAsStream("/WEB-INF/client_secrets.json");

For some reason, AppEngine won't let you load resources under WEB-INF using the class loader (you can load resources in jar files with the above technique).

You can use the File API to access paths in WEB-INF in App, but the paths need to be relative. Assuming that the file client_secrets.json is in the WEB-INF folder, this should work:

clientSecrets = GoogleClientSecrets.load(JSON_FACTORY,
    new InputStreamReader(new FileInputStream(
        new File("WEB-INF/client_secrets.json"))));

To use the File APIs to read resources under WEB-INF, you also also need to make sure your appengine-web.xml file includes the resources in your WEB-INF:

<resource-files>
  <include path="/**.json" />
</resource-files>

See Configuring appengine-web.xml and Why can't I read from this file