Google App Engine Error: Fetch in a thread that is neither the original request thread nor a thread created by ThreadManager

Question

App Engine gives the error:

com.google.apphosting.api.ApiProxy$CallNotFoundException: Can't make API call urlfetch.Fetch in a thread that is neither the original request thread nor a thread created by ThreadManager

when I make call to Google Vision API inside Callable in async Servlet. How to make it working?

servlet:

public class OcrForTextServlet extends HttpServlet {
    @Override
    public void doPost(HttpServletRequest req, HttpServletResponse response) {
            byte[] file = extractFile(req);
            String[] languages = req.getParameterValues("language");
            ExecutorService executor = Executors.newFixedThreadPool(2);
            Future<String> result = executor.submit(new OcrCallableTask(file, languages));
            executor.shutdown();
            response.getWriter().write(result.get()); //ERROR HERE

Full stack trace:

[INFO] java.util.concurrent.ExecutionException: com.google.apphosting.api.ApiProxy$CallNotFoundException: Can't make API call urlfetch.Fetch in a thread that is neither the original request thread nor a thread created by ThreadManager
[INFO]  at java.util.concurrent.FutureTask.report(FutureTask.java:122)
[INFO]  at java.util.concurrent.FutureTask.get(FutureTask.java:192)
[INFO]  at ocrme_backend.servlets.ocr.OcrForTextServlet.doPost(OcrForTextServlet.java:49)
[INFO]  at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
[INFO]  at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
[INFO]  at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
[INFO]  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
[INFO]  at com.google.appengine.api.socket.dev.DevSocketFilter.doFilter(DevSocketFilter.java:74)
[INFO]  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[INFO]  at com.google.appengine.tools.development.ResponseRewriterFilter.doFilter(ResponseRewriterFilter.java:134)
[INFO]  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[INFO]  at com.google.appengine.tools.development.HeaderVerificationFilter.doFilter(HeaderVerificationFilter.java:34)
[INFO]  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[INFO]  at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:63)
[INFO]  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[INFO]  at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:48)
[INFO]  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[INFO]  at com.google.appengine.tools.development.jetty9.StaticFileFilter.doFilter(StaticFileFilter.java:122)
[INFO]  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[INFO]  at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectRequest(DevAppServerModulesFilter.java:366)
[INFO]  at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectModuleRequest(DevAppServerModulesFilter.java:349)
[INFO]  at com.google.appengine.tools.development.DevAppServerModulesFilter.doFilter(DevAppServerModulesFilter.java:116)
[INFO]  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751)
[INFO]  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[INFO]  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
[INFO]  at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
[INFO]  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
[INFO]  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[INFO]  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[INFO]  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[INFO]  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[INFO]  at com.google.appengine.tools.development.jetty9.DevAppEngineWebAppContext.doScope(DevAppEngineWebAppContext.java:112)
[INFO]  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[INFO]  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[INFO]  at com.google.appengine.tools.development.jetty9.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:596)
[INFO]  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[INFO]  at org.eclipse.jetty.server.Server.handle(Server.java:534)
[INFO]  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[INFO]  at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[INFO]  at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[INFO]  at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[INFO]  at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[INFO]  at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[INFO]  at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[INFO]  at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[INFO]  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[INFO]  at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[INFO]  at java.lang.Thread.run(Thread.java:745)
[INFO] Caused by: com.google.apphosting.api.ApiProxy$CallNotFoundException: Can't make API call urlfetch.Fetch in a thread that is neither the original request thread nor a thread created by ThreadManager
[INFO]  at com.google.apphosting.api.ApiProxy$CallNotFoundException.foreignThread(ApiProxy.java:844)
[INFO]  at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:116)
[INFO]  at com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:40)
[INFO]  at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.fetchResponse(URLFetchServiceStreamHandler.java:543)
[INFO]  at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.getInputStream(URLFetchServiceStreamHandler.java:422)
[INFO]  at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.getResponseCode(URLFetchServiceStreamHandler.java:275)
[INFO]  at com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:37)
[INFO]  at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:94)
[INFO]  at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:972)
[INFO]  at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:283)
[INFO]  at com.google.api.client.auth.oauth2.TokenRequest.execute(TokenRequest.java:307)
[INFO]  at com.google.api.client.auth.oauth2.Credential.executeRefreshToken(Credential.java:570)
[INFO]  at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.executeRefreshToken(GoogleCredential.java:362)
[INFO]  at com.google.api.client.auth.oauth2.Credential.refreshToken(Credential.java:489)
[INFO]  at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.fromStreamUser(GoogleCredential.java:772)
[INFO]  at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.fromStream(GoogleCredential.java:257)
[INFO]  at com.google.api.client.googleapis.auth.oauth2.DefaultCredentialProvider.getCredentialUsingWellKnownFile(DefaultCredentialProvider.java:249)
[INFO]  at com.google.api.client.googleapis.auth.oauth2.DefaultCredentialProvider.getDefaultCredentialUnsynchronized(DefaultCredentialProvider.java:117)
[INFO]  at com.google.api.client.googleapis.auth.oauth2.DefaultCredentialProvider.getDefaultCredential(DefaultCredentialProvider.java:91)
[INFO]  at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.getApplicationDefault(GoogleCredential.java:213)
[INFO]  at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.getApplicationDefault(GoogleCredential.java:191)
[INFO]  at ocrme_backend.ocr.OCRProcessorImpl.getVisionService(OCRProcessorImpl.java:40)
[INFO]  at ocrme_backend.ocr.OCRProcessorImpl.<init>(OCRProcessorImpl.java:32)
[INFO]  at ocrme_backend.servlets.ocr.OcrCallableTask.doStaff(OcrCallableTask.java:27)
[INFO]  at ocrme_backend.servlets.ocr.OcrCallableTask.call(OcrCallableTask.java:39)
[INFO]  at ocrme_backend.servlets.ocr.OcrCallableTask.call(OcrCallableTask.java:14)
[INFO]  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO]  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[INFO]  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[INFO]  ... 1 more

API call makes the error:

/**
 * Connects to the Vision API using Application Default Credentials.
 */
public static Vision getVisionService() throws IOException, GeneralSecurityException {
    GoogleCredential credential =
            GoogleCredential.getApplicationDefault().createScoped(VisionScopes.all());
    com.google.api.client.json.JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
    return new Vision.Builder(GoogleNetHttpTransport.newTrustedTransport(), jsonFactory, credential)
            .setApplicationName(APPLICATION_NAME)
            .build();
}

I am using the last version of javax.servlet-api (3.1.0), GAE (1.9.52) and Java 8. I need to obtain the result from the async part. How can I do this? Thank you for any help.

UPDATE:
I tried to use com.google.appengine.api.ThreadManager as mentioned in error message but it gives the same error. Here is my updated servlet:

@Override
public void doPost(HttpServletRequest req, HttpServletResponse response) {

    try {
        byte[] file = extractFile(req);
        String[] languages = req.getParameterValues("language");

        ThreadFactory factory = ThreadManager.currentRequestThreadFactory();
        ExecutorService service = Executors.newCachedThreadPool(factory);
        Future<String> result =
                service.submit(new OcrCallableTask(file, languages));
        response.getWriter().write(result.get()); //ERROR HERE

Next test passed OK:

public class OcrCallableTaskTest {
    @Test
    public void testCall() throws Exception {
        ExecutorService service = Executors.newFixedThreadPool(2);
        Future<String> result = service.submit(new OcrCallableTask(FileProvider.getFile(), null));
        Assert.assertTrue(result.get() != null);
        Assert.assertTrue(result.get().length() > 0);
    }
}

UPDATE 2:
(Reply to the proposing do staff in request's thread.)
Realy I don't need extra thread for my servlet. It is only the attempt to fix the error. I have the same error if I don't use multithreading in my app:

[INFO] com.google.api.gax.grpc.ApiException: io.grpc.StatusRuntimeException: UNAUTHENTICATED
[INFO]  at com.google.api.gax.grpc.ExceptionTransformingCallable$ExceptionTransformingFuture.onFailure(ExceptionTransformingCallable.java:108)
[INFO]  at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:52)
[INFO]  at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310)
[INFO]  at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
[INFO]  at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
[INFO]  at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
[INFO]  at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
[INFO]  at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:463)
[INFO]  at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:439)
[INFO]  at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:428)
[INFO]  at io.grpc.internal.ClientCallImpl.access$100(ClientCallImpl.java:76)
[INFO]  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:514)
[INFO]  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700(ClientCallImpl.java:431)
[INFO]  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:546)
[INFO]  at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:52)
[INFO]  at io.grpc.internal.SerializingExecutor$TaskRunner.run(SerializingExecutor.java:152)
[INFO]  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[INFO]  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO]  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[INFO]  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[INFO]  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[INFO]  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[INFO]  at java.lang.Thread.run(Thread.java:745)
[INFO] Caused by: io.grpc.StatusRuntimeException: UNAUTHENTICATED
[INFO]  at io.grpc.Status.asRuntimeException(Status.java:540)
[INFO]  ... 15 more
[INFO] Caused by: com.google.apphosting.api.ApiProxy$CallNotFoundException: Can't make API call urlfetch.Fetch in a thread that is neither the original request thread nor a thread created by ThreadManager
[INFO]  at com.google.apphosting.api.ApiProxy$CallNotFoundException.foreignThread(ApiProxy.java:844)
[INFO]  at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:116)
[INFO]  at com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:40)
[INFO]  at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.fetchResponse(URLFetchServiceStreamHandler.java:543)
[INFO]  at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.getInputStream(URLFetchServiceStreamHandler.java:422)
[INFO]  at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.getResponseCode(URLFetchServiceStreamHandler.java:275)
[INFO]  at com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:37)
[INFO]  at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:94)
[INFO]  at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:972)
[INFO]  at com.google.auth.oauth2.UserCredentials.refreshAccessToken(UserCredentials.java:207)
[INFO]  at com.google.auth.oauth2.OAuth2Credentials.refresh(OAuth2Credentials.java:149)
[INFO]  at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:135)
[INFO]  at io.grpc.auth.GoogleAuthLibraryCallCredentials$1.run(GoogleAuthLibraryCallCredentials.java:110)
[INFO]  ... 7 more

code of servlet - no multithreading:

 @Override
    public void doPost(HttpServletRequest req, HttpServletResponse response) {
        try {
            byte[] file = extractFile(req);
            String[] languages = req.getParameterValues("language");

            OCRProcessor processor = new OCRProcessorImpl();
            String jsonResult;

            if (languages == null || languages.length <= 0) { 
                jsonResult = processor.ocrForText(file);

next test passed ok:

@Test
public void doOCR() throws Exception {
    byte[] file = FileProvider.getImageFile().getFile();
    String result = ocrProcessor.ocrForText(file);
    assertNotNull(result);
    assertTrue(result.length() > 0);
}

It seams (GAE + API calls) not compatible with Servlet architecture. Thank you for any advices.

Answer 1

Async?

Maybe I misunderstand you, but your servlet seems not async:

    Future<String> result =
            service.submit(new OcrCallableTask(file, languages));
    response.getWriter().write(result.get()); //ERROR HERE

You servlet thread will block until this url fetch finish because Future#get() will wait until task complete.

And, as your stacktrace shows, the ApiProxy also invoked the snyc method:

at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:116)

Suggestions

It seems that Google App Engine still not open sourced, so I can just give your some possible directions:

• From the doc of ApiProxy, this class may using some variable like ThreadLocal to store data, so creating new thread using Executors may confused App Engine;

  It also stores an Environment for each thread, which contains additional user-visible information about the request

• In your case, I didn't see the necessity of using multiple thread, you may would like to try fetch info using request thread;

• If you indeed need async call, you may need using background thread, then notify user when call is finished;

---

More

I try to run your example, but your Vision class fails to compile. So I write up another example, here, which runs fine.

The following is the source code decompiled from sdk where exception happends:

    ApiProxy.Environment threadLocalEnvironment = (ApiProxy.Environment)environmentThreadLocal.get();
    if(threadLocalEnvironment != null) {
        return threadLocalEnvironment;
    } else {
        ApiProxy.EnvironmentFactory envFactory = getEnvironmentFactory();
        if(envFactory != null) {
            ApiProxy.Environment environment = envFactory.newEnvironment();
            environmentThreadLocal.set(environment);
            return environment;
        } else {
            return null;
        }
    }

In your case, because it fails to get the environment factory, then it fails to get the environment, which leads to your exception.

When it comes to why it lack environment factory, I guess you made wrong configuration (do you run your war with GAE's server? if you test locally, it is necessary.)