Push to google container registry fails: Retrying

Question

I'm trying to push to the Google container registry from my Jenkins. The builds run inside the Kubernetes Jenkins Plugin, which uses the gcr.io/cloud-solutions-images/jenkins-k8s-slave to build the docker image into the Kubernetes native Docker.

After authenticating to the Google container registry I'm trying to push the newly built image. This is my pipeline script:

def imageTag = 'gcr.io/project-id/tag'

def version = version from pom 

sh './mvnw package'

sh "docker build -t $imageTag:$version ."

sh('gcloud auth activate-service-account --key-file=$FILE')

sh('docker login -p $(gcloud auth print-access-token) -u _token https://gcr.io')

sh("gcloud docker -- push $imageTag:$version")

The push fails with the following output:

c6ff94654483: Preparing
209db64c273a: Preparing
762429e05518: Preparing
2be465c0fdf6: Preparing
5bef08742407: Preparing
c6ff94654483: Retrying in 5 seconds
5bef08742407: Retrying in 5 seconds
209db64c273a: Retrying in 5 seconds
2be465c0fdf6: Layer already exists
762429e05518: Layer already exists
c6ff94654483: Retrying in 4 seconds
5bef08742407: Retrying in 4 seconds
209db64c273a: Retrying in 4 seconds
c6ff94654483: Retrying in 3 seconds
5bef08742407: Retrying in 3 seconds
209db64c273a: Retrying in 3 seconds
c6ff94654483: Retrying in 2 seconds
5bef08742407: Retrying in 2 seconds
209db64c273a: Retrying in 2 seconds
c6ff94654483: Retrying in 1 second
5bef08742407: Retrying in 1 second
209db64c273a: Retrying in 1 second
5bef08742407: Retrying in 10 seconds
...
unexpected EOF

Answer 1

The root cause of this issue is that your docker daemon is not authenticated with the credentials necessary to push to gcr.io. For the original question, I believe this is likely because the user account being used was _token instead of oauth2accesstoken.

I was experiencing an error similar to this, except that instead of using docker login, I was using docker-credential-gcr and was getting the same unexpected EOF error.

My problem was the fact that I was running on GCE, from which docker-credential-gcr was detecting and using a different service account via the GCE metadata API.

So, for others experiencing this issue who are running on GCP and trying to authenticate a service account via docker-credential-gcr, you need to tell it to only look at the gcloud credentials, instead of looking at the environment for the metadata API details. My flow looks like this now:

gcloud auth activate-service-account --key-file=$FILE

docker-credential-gcr configure-docker --token-source="gcloud"

docker push gcr.io/....

Hope it helps someone.

Answer 2

Check if you use correct projectID in tag as it was solved in Cannot push image to repository in Google Container Engine