AWS Lambda Timeout when making external HTTPS Request

Question

I need to use Firebase Admin Java in my AWS Lambda project, but Firebase Auth Admin throws a SocketTimeoutException when it tries to fetch a Public Key within the Google's servers. I know Firebase Admin is configured correctly, because I can run it locally in a Tomcat instance with no worries.

I tried then to just make an HTTPS Request to a test API like https://postman-echo.com/get?foo1=bar1&foo2=bar2, and it has the same behavior, I get a timeout as well, which leads me to think that this is a problem with Lambda invoking an HTTPS Request.

Now what I know:

1. My lambda is integrated with API Gateway
2. My lambda is inside a subnet with internet access through an Internet Gateway
3. My lambda's Security Group is wide open
4. I know this problem happens on HTTPS Requests, I am not sure about pure HTTP. This is hard to test because I can't find an API out there that will not automatically redirect from HTTP to HTTPS, and I can't roll my own right now.

Just for the sake of completeness I will leave here the SocketTimeoutException I get from Firebase Admin Java, although I am not sure yet if this is a problem within the Firebase Admin library or with AWS Lambda + HTTPS or both together:

[main] ERROR com.example.project.filters.FirebaseAuthenticationFilter - Error while parsing or validating the token com.google.firebase.auth.FirebaseAuthException: Error while verifying signature. at com.google.firebase.auth.FirebaseTokenVerifierImpl.checkSignature(FirebaseTokenVerifierImpl.java:160) at com.google.firebase.auth.FirebaseTokenVerifierImpl.verifyToken(FirebaseTokenVerifierImpl.java:92) at com.google.firebase.auth.FirebaseAuth$4.execute(FirebaseAuth.java:426) at com.google.firebase.auth.FirebaseAuth$4.execute(FirebaseAuth.java:423) at com.google.firebase.internal.CallableOperation.call(CallableOperation.java:36) at com.google.firebase.auth.FirebaseAuth.verifyIdToken(FirebaseAuth.java:388) at com.google.firebase.auth.FirebaseAuth.verifyIdToken(FirebaseAuth.java:362) at com.example.project.filters.FirebaseAuthenticationFilter.filter(FirebaseAuthenticationFilter.java:55) at org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:132) at org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:68) at org.glassfish.jersey.process.internal.Stages.process(Stages.java:197) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at com.amazonaws.serverless.proxy.jersey.JerseyHandlerFilter.doFilter(JerseyHandlerFilter.java:91) at com.amazonaws.serverless.proxy.internal.servlet.FilterChainHolder.doFilter(FilterChainHolder.java:84) at com.amazonaws.serverless.proxy.internal.servlet.AwsLambdaServletContainerHandler.doFilter(AwsLambdaServletContainerHandler.java:206) at com.amazonaws.serverless.proxy.jersey.JerseyLambdaContainerHandler.handleRequest(JerseyLambdaContainerHandler.java:184) at com.amazonaws.serverless.proxy.jersey.JerseyLambdaContainerHandler.handleRequest(JerseyLambdaContainerHandler.java:76) at com.amazonaws.serverless.proxy.internal.LambdaContainerHandler.proxy(LambdaContainerHandler.java:177) at com.amazonaws.serverless.proxy.internal.LambdaContainerHandler.proxyStream(LambdaContainerHandler.java:209) at com.example.project.StreamLambdaHandler.handleRequest(StreamLambdaHandler.java:39) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at lambdainternal.EventHandlerLoader$StreamMethodRequestHandler.handleRequest(EventHandlerLoader.java:350) at lambdainternal.EventHandlerLoader$2.call(EventHandlerLoader.java:888) at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:293) at lambdainternal.AWSLambda.(AWSLambda.java:64) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at lambdainternal.LambdaRTEntry.main(LambdaRTEntry.java:114) Caused by: java.net.SocketTimeoutException: connect timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:666) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) at sun.net.www.protocol.https.HttpsClient.(HttpsClient.java:264) at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191) at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177) at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162) at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:104) at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981) at com.google.api.client.googleapis.auth.oauth2.GooglePublicKeysManager.refresh(GooglePublicKeysManager.java:172) at com.google.api.client.googleapis.auth.oauth2.GooglePublicKeysManager.getPublicKeys(GooglePublicKeysManager.java:140) at com.google.firebase.auth.FirebaseTokenVerifierImpl.isSignatureValid(FirebaseTokenVerifierImpl.java:226) at com.google.firebase.auth.FirebaseTokenVerifierImpl.checkSignature(FirebaseTokenVerifierImpl.java:152) ... 38 more

Answer 1

My lambda is inside a subnet with internet access through an Internet Gateway

An Internet Gateway alone will not provide a Lambda function residing in a VPC access to the Internet. This is because Lambda functions inside a VPC do not receive public IP addresses. You have to use a NAT Gateway to provide Internet Access to a Lambda function inside a VPC.