are updated health checks causing App Engine deployment to fail?

Question

we updated our google app engine health checks from the legacy version to the new version using and now our deployments are failing. Nothing else on the project has changed. We tested the default settings and then extended checks just in case.

This is the error: ERROR: (gcloud.app.deploy) Error Response: [4] Your deployment has failed to become healthy in the allotted time and therefore was rolled back. If you believe this was an error, try adjusting the 'app_start_timeout_sec' setting in the 'readiness_check' section.

This is our app.yaml:

liveness_check:
   check_interval_sec: 120
   timeout_sec: 40
   failure_threshold: 5
   success_threshold: 5
   initial_delay_sec: 500

readiness_check:
  check_interval_sec: 120
  timeout_sec: 40
  failure_threshold: 5
  success_threshold: 5
  app_start_timeout_sec: 1500

Unfortunately, no matter the configuration, both the readiness and liveness checks are throwing 404s.

What could be causing the problem? and how can we debug this? Is it possible to rollback to the legacy health checks?

Answer 1

In my case, I solved this issue by manually increasing memory allocation?

resources:
    cpu: 1
    memory_gb: 2
    disk_size_gb: 10

Found this solution in a google forum: https://groups.google.com/forum/#!topic/google-appengine/Po_-SkC5DOE

Answer 2

This is usually caused when the application is still reading from the legacy health check flags and/or deploying the app using gcloud app deploy without enabling the updated health checks first. You can check this by:

1- Making sure the legacy health_check flag does not exist on your app.yaml.

2- Run gcloud beta app describe to see whether splitHealthChecks flag is set to true under featureSettings.

By default, HTTP requests from updated health checks are not forwarded to your application container. If you want to extend health checks to your application, then specify a path for liveness checks or readiness checks.

You can then enable updated health checks by using gcloud beta app update --split-health-checks --project [your-project-id]. See this public issue tracker or this article about Updated Health Checks about for more details.