GCP Cloud build ignores timeout settings

Question

I use Cloud Build for copying the configuration file from storage and deploying the app to App Engine flex. The problem is that the build fails every time when it lasts more than 10 minutes. I've specified timeout in my cloudbuild.yaml but it looks like it's ignored. Also, I configured app/cloud_build_timeout and set it to 1000. Could somebody explain to me what is wrong here?

My cloudbuild.yaml looks in this way:

steps:
  - name: gcr.io/cloud-builders/gsutil
    args: ["cp", "gs://myproj-dev-247118.appspot.com/.env.cloud", ".env"]
  - name: "gcr.io/cloud-builders/gcloud"
    args: ["app", "deploy"]
    timeout: 1000s
timeout: 1600s

My app.yaml use custom env that build it from Dockerfile and looks like this:

runtime: custom
env: flex

manual_scaling:
  instances: 1
env_variables:
  NODE_ENV: dev

Dockerfile also contains nothing special, just installing dependencies and app building:

FROM node:10 as front-builder
WORKDIR /app
COPY front-end .
RUN npm install
RUN npm run build:web

FROM node:12
WORKDIR /app
COPY api .
RUN npm install
RUN npm run build
COPY .env .env
EXPOSE 8080
COPY --from=front-builder /app/web-build web-build
CMD npm start

Answer 1

When running gcloud app deploy directly for an App Engine Flex app, from your local machine for example, under the hood it spawns a Cloud Build job to build the image that is then deployed to GAE (you can see that build in Cloud Console > Cloud Build). This build has a 10min timeout that can be customized via:

gcloud config set app/cloud_build_timeout 1000

Now, the issue here is that you're issuing the gcloud app deploy command from within Cloud Build itself. Since each individual Cloud Build step is running in its own Docker container, you can't just add a previous step to customize the timeout since the next one will use the default gcloud setting.

You've got several options to solve this:

• Add a build step to first build the image with docker build, upload it to Google Cloud Registry. You can set a custom timeout on these steps to fit your needs. Finally, deploy your app with glcoud app deploy --image-url=IMAGE-URL.
• Create your own custom gcloud builder where app/cloud_build_timeout is set to your custom value. You can derive it from the default gcloud builder Dockerfile and add /builder/google-cloud-sdk/bin/gcloud config set app/cloud_build_timeout 1000

Answer 2

Just in case if you are using Google Cloud Build with Skaffold, remember checking the skaffold.yaml if you setted the timeout option inside the googleCloudBuild section in build. For example:

build: 
  googleCloudBuild: 
    timeout: 3600s

Skaffold will ignore the gcloud config of the machine where you are running the deploy. For example it will ignore this CLI command: gcloud config set app/cloud_build_timeout 3600