Google is blocking traffic because it detects automated queries

Question

When I try to access my website deployed on Google Cloud I receive the following error page:

We're sorry...

... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.

Here are the details of my setup:

• NextJS web application deployed to Cloud Run (traffic limited to Internal + Load Balancing)
• Strapi Content Management System deployed to Cloud Run (traffic limited to Internal + Load Balancing)
• I have a load balancer in front of each Cloud Run service and am using Google reserved IP's and managed SSL certs
• Identity Aware Proxy (IAM) is in front of both Load Balancers
• Each page visit makes an API call to the content management system to retrieve content (could this be causing the network to think "automated queries" are occurring?)

Troubleshooting:

• It doesn't appear to be related to the actual user visiting the website. This error message appears for all users, different devices, different IP addresses. It was working without issue before, this is the first time I've had this problem.
• No traffic is reaching the Cloud Run environments. If I switch their ingress to Allow All I am able to hit the Cloud Run URL directly and it works
• Deleting and recreating the load balancer doesn't resolve the issue

Based on this, there seems to be some network configuration/security occurring near the load balancer.

Any solutions or suggestions to continue debugging?

Answer 1

So far the two only workarounds available are:

• disable IAP on the load balancer
• Make Public and Access directly the Cloud RUN URL (https://myCloudRun-xxxxx.a.run.com)

Both are not acceptable on production systems unless you don't care about security. We are trying to work out what are the IAP headers lost between NGINX and Cloud RUN but I guess the issue affects also infrastructure without NGINX

EDIT the issue tracker post https://issuetracker.google.com/issues/265184232 looks related to the issue. The workaround suggested by the google team is to switch to a Global HTTP Classic Load Balancer We tried and it did not work but we may have a different problem. Hope that helps anyone else for the time being

EDIT 2 In our NGINX configuration we proxy the request to a public Cloud RUN API which at the moment is affected by the issue.

I was able to proxy the request successfully just by removing the JWT header added by IAP (leaving any other headers with the authenticated user) example of my config

 location /api/ {
    #remove iap jwt token header 
    proxy_set_header        X-Goog-Iap-Jwt-Assertion "";

    # proxy to the public cloud run instance
    proxy_pass https://myCloudRun-123456basd-ew.a.run.app/;
}