Receiving "SSL Unexpected EOF" error in Google Cloud Run when performing GET API request in code

Question

I am configuring my application in Google Cloud Run. My app performs several API calls to an external service via GET and POST requests. My GET request fails on the cloud only, while working from the same container on my computer. I'm getting:

SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1007)

It might be a Cloud Run configuration issue with handling incoming responses from requests, but I haven't found a way to configure it correctly.
My code is written in python 3.10 and my app is running using FastAPI. I'm using Google Cloud Run with Google Cloud Build to build my container. POST API calls work correctly.

Edit: I have found that these GET API calls work the first times they are called but fail with the SSL Error the next time:

app = FastAPI()

@app.on_event("startup")
async def init() -> None:
    """ Initialize varibles to start app """    
    if co.DEV:
        res = whatsapp.send_msg_to_test_group("DEV Deployment is Up!", co.TEST_WHATSAPP_GROUP_ID)
        logging.info(res)    
    else:
        logging.info("Production is Up!")


@app.on_event("startup")
@repeat_every(seconds=60, wait_first=False, logger=toko_logger, raise_exceptions=co.RAISE_EXCEPTIONS)
async def test_api() -> None:
    import requests
    for i in range(3):
        resp = requests.request("GET", url=<MY_URL>, headers=<HEADERS>)                
        logger.debug(f"details: {len(resp.json())}")
        sleep(10)

Meaning that the first time test_api() is called I see 3 successful logs, but on the second time I get the SSL exception. I cannot provide my URL or headers as they involve my personal API token.

And just to emphasize, when I run the same docker on my personal computer the API call is successful every call to test_api().

my requirements.txt for this example:

fastapi==0.104.0
fastapi-utils==0.2.1
requests==2.31.0
uvicorn==0.24.0.post1

My docker file run command is:

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080", "--reload"]

• The API call works from python on a VM on the cloud.
• The same error on Cloud Run occurs when I set verify=False in the request.

Any ideas on why this is happening and what can be done?

Answer 1

I was seeing the same error when using a package that makes requests to an external API. My set up is also FastAPI on Cloud Run and I had no issues for months, until Nov 25.

I had my requests running in BackgroundTasks on a throttled Cloud Run instance, and have read previously that background processes in this set up do not keep the Cloud Run instance running. So my guess is that the instance was idling before the request could finish nicely.

I've set my instance to be --no-cpu-throttling and haven't seen any issues since but will keep monitoring.

(btw I'm using httpx library, not requests)

Answer 2

Solution: Use the httpx module instead of requests.

Changes to requirements.txt:

pip install httpx

Changes to code:

# requires.post(...)
httpx.post(...)
# You may need to update your exceptions as well.

Strangely, in my case, I only had a problem when using the requests library to communicating with Cloud Run. Even communicating with BigQuery was no problem.

(A comment by @sokeefe here on Stackoverflow tipped me off to this workaround.)