Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Handling a Cloud Run container shutdown

When writing a Cloud Run service, we develop a container that listens on the PORT environment variable to handle an incoming HTTP request. An instance of the container springs to life and handles the request and then lives a while longer after concluding the original request in case there are further requests to arrive. If there are no further requests, GCP shuts down the container. It is in this area that I have a question.

Is there a hook, signal or other indication that the container is being shut down within the container?

In my example, my container wants to cleanly end. Perhaps it wants to close connections or perform some quick flush of cache.

like image 571
Kolban Avatar asked Dec 26 '19 16:12

Kolban


People also ask

How do I stop cloud run container?

Cloud Run does not offer a direct way to make a service stop serving traffic, but you can achieve a similar result by revoking the permission to invoke the service to identities that are invoking the service. Notably, if your service is "public", remove allUsers from the Cloud Run Invoker role ( roles/run.

Can cloud functions run containers?

Cloud Build then automatically builds your code into a container image and pushes that image to a image registry (either Container Registry or Artifact Registry). Cloud Functions accesses this image when it needs to run the container to execute your function.

How long do cloud run containers last?

To minimize the impact of cold starts, Cloud Run may keep some instances idle for a maximum of 15 minutes. These instances are ready to handle requests in case of a sudden traffic spike.


2 Answers

An FAQ of great collected Q&A on Cloud Run has been found here. Within that FAQ there is an item which reads:

What is the termination signal for Cloud Run services?

Currently, Cloud Run terminates containers while scaling to zero with unix signal 9 (SIGKILL). SIGKILL is not trappable (capturable) by applications. Therefore, your applications should be okay to be killed abruptly.

A related and important entry also reads:

When will my service scale to zero?

Cloud Run does not provide any guarantees on how long it will keep a service "warm". It depends on factors like capacity and Google’s implementation details.

Some users see their services staying warm up to an hour, or longer.


Opinion

I find it interesting that the story appears to be an immediate SIGKILL. If we take Docker as a basis for a container environment, we can read about docker stop which appears to be the way to cleanly stop a container. In its own description it says:

The main process inside the container will receive SIGTERM, and after a grace period, SIGKILL.

This would appear to indicate that for a normal Docker container stop, the process running the container will receive a signal.

like image 138
Kolban Avatar answered Sep 20 '22 16:09

Kolban


SIGTERM (unix signal 15) is now sent to the Cloud Run service and you are given 10 seconds to handle closing any resources etc, if you handle the signal in your code.

See here.

Container instances can be shut down at any time. When a container instance needs to be shut down, new incoming requests are routed to other instances and requests currently being processed are given time to complete. The container instance then receives a SIGTERM signal indicating the start of a 10 second period before being shut down (with a SIGKILL signal). During this period, the container instance is allocated CPU and billed. If the container instance does not catch the SIGTERM signal, it is immediately shut down.

enter image description here

(image of code taken from here)

like image 32
Graham Polley Avatar answered Sep 16 '22 16:09

Graham Polley