Menu
So after googling a little bit (which is polluted by people having trouble with Pull Secrets) I am posting this here — and to GCP Support (will update as I hear).
I created a Cluster from GitLab Kubernetes integration (docs: https://about.gitlab.com/solutions/kubernetes) within the same project as my GCR registry / images.
When I add a new service / deployment to this Cluster using Kubectl (which relies on a private image within the GCR Registry in this project) the pods in the GitLab created cluster fail to pull from GCR with: ErrImagePull.
To be clear — I am NOT pulling from a GitLab private registry, I am attempting to pull from a GCR Registry within the same project as the GKE cluster created from GitLab (which should not require a Pull Secret).
Other Clusters (created from GCP console) within this project can properly access the same image so my thinking is that there is some difference between Clusters created via an API (in this case from GitLab) vs Clusters created from the GCP console.
I am hoping someone has run into this in the past — or can explain the differences in the Service Accounts etc that could be causing the problem.
I am going to attempt to create a service account and manually grant it Project Viewer role to see if that solves the problem.
Update: manually configured Service Account did not solve issue.
Note: I am trying to pull an image into the Cluster NOT into a GitLab Runner that is running on the Cluster. Ie. I want a separate Service / Deployment to be running along side my GitLab infrastructure.
333
To connect a Kubernetes cluster to GitLab, you must first install an agent in your cluster. The agent runs in the cluster, and you can use it to: Communicate with a cluster, which is behind a firewall or NAT. Access API endpoints in a cluster in real time.
Connect to the cluster from your own computer You can easily connect to your cluster and run Kubernetes commands from your own terminal. Before using the same command line access displayed above, you need to have installed: Google Cloud CLI from the Google SDK. kubectl , which is the Kubernetes command line interface.
If your version of GKE is earlier than the minimum version, you must configure Kubernetes imagePullSecrets to enable GKE to pull images. If GKE is in a different project than Artifact Registry, grant Artifact Registry permissions to the service account that your GKE node use. By default, nodes use the Compute Engine default service account.
Today I Learned: Pull Docker Image from GCR (Google Container Registry) in any non-GCP Kubernetes… A simple guide to help you use GCR as your Container Registry in any non-GCP Kubernetes Cluster. The issue is about authentication to GCR when pulling the private images.
But, I just migrate the Kubernetes clusters and Database. And I still used a few services from GCP, for example, Google services like GCR (Google Container Registry) for my container registry, because GCR is a bit cheap compared to DockerHub for the private registry.
The issue is about authentication to GCR when pulling the private images. The guide explains how to a pull Docker Image from GCR in any other non-Google Cloud Platform Kubernets cluster using GCR. GCR is a bit cheap compared to DockerHub for the private registry.
TL;DR — Clusters created by GitLab-Ci Kubernetes Integration will not be able to pull an image from a GCR Registry in the same project as the container images — without modifying the Node(s) permissions (scopes).
By default the Cluster Nodes created by a Cluster which was itself created by GitLab-Ci's Kubernetes Integration are created with minimal permissions (scopes) to Google Cloud services.
You can see this visually from the GCP console dashboard for your cluster, scroll down to the permissions section and look for "Storage":
This essentially means that the Node(s) running within your GitLab-Ci Kubernetes integration cluster WILL NOT have the default GCR Registry (read-only) permissions necessary to pull an image from a GCR Registry.
It also means (as far as I can tell) that even if you grant a Service Account proper access to the GCR Registry it still will not work — not totally sure I set my Service Account up properly but I believe I did.
Great.
Basically you have two options. The first one is to create a Cluster (ie. outside of GitLab Kubernetes Integration) and then re-connect your GitLab project to THAT Cluster by following the manual 'connect to an existing Cluster' directions that can be found here: https://gitlab.com/help/user/project/clusters/index#adding-an-existing-kubernetes-cluster
The second option is to modify your permissions but that is more complicated.
192
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
