Google Cloud Build deploy to GKE Private Cluster

Question

I'm running a Google Kubernetes Engine with the "private-cluster" option. I've also defined "authorized Master Network" to be able to remotely access the environment - this works just fine. Now I want to setup some kind of CI/CD pipeline using Google Cloud Build - after successfully building a new docker image, this new image should be automatically deployed to GKE. When I first fired off the new pipeline, the deployment to GKE failed - the error message was something like: "Unable to connect to the server: dial tcp xxx.xxx.xxx.xxx:443: i/o timeout". As I had the "authorized master networks" option under suspicion for being the root cause for the connection timeout, I've added 0.0.0.0/0 to the allowed networks and started the Cloud Build job again - this time everything went well and after the docker image was created it was deployed to GKE. Good.

The only problem that remains is that I don't really want to allow the whole Internet being able to access my Kubernetes master - that's a bad idea, isn't it?

Are there more elegant solutions to narrow down access by using allowed master networks and also being able to deploy via cloud build?

Answer 1

It's currently not possible to add Cloud Build machines to a VPC. Similarly, Cloud Build does not announce IP ranges of the build machines. So you can't do this today without creating a "ssh bastion instance" or a "proxy instance" on GCE within that VPC.

I suspect this would change soon. GCB existed before GKE private clusters and private clusters are still a beta feature.

Answer 2

We ended up doing the following:

1) Remove the deployment step from cloudbuild.yaml

2) Install Keel inside the private cluster and give it pub/sub editor privileges in the cloud builder / registry project

Keel will monitor changes in images and deploy them automatically based on your settings.

This has worked out great as now we get pushed sha hashed image updates, without adding vms or doing any kind of bastion/ssh host.