Docker Python library cannot properly login into AWS ECR

Question

I wrote a Python script to pull images from AWS ECR to an Ubuntu instance. On this instance, I run docker commands with sudo as Docker is not setup as a non-root user.

I do use sudo when invoking the script. What I find is if I am currently logged into AWS ECR first and then I run the script, it works as expected. However, if I am not logged in, and the auth token is expired, it appears that docker login works, but when I try and pull I get a message indicating that the "repository does not exist or my require 'docker login'".

Examining logs, it verifies this with

Feb 15 06:00:38 ubuntu-xenial dockerd[1388]: time="2019-02-15T06:00:38.832827449Z" level=error msg="Not continuing with pull after error: denied: Your Authorization Token has expired. Please run 'aws ecr get-login --no-include-email' to fetch a new one."

def log_into_aws_ecr(docker_client, region):
    # To do, set region
    ecr_client = boto3.client('ecr', region_name=region)

    # Get all repos
    response = ecr_client.describe_repositories()
    repo_names = []
    repositories = response.get('repositories', [])
    for repo in repositories:
        name = repo.get('repositoryName', '')
        if len(name):
            repo_names.append(name)
    token = ecr_client.get_authorization_token()
    username, password = base64.b64decode(token['authorizationData'][0]['authorizationToken']).decode('utf-8').split(":")
    registry_url = token['authorizationData'][0]['proxyEndpoint']
    login_results = docker_client.login(username, password, email='', registry=registry_url)

    prefix='https://'
    if registry_url.startswith(prefix):
        registry = registry_url[len(prefix):]
    else:
        registry = registry_url
    auth_config_payload = {'username': username, 'password': password }
    return ecr_client, repo_names, registry

Please note that this code is also being re-factored now, so there are some variables that are defined but not currently in use.

The supplied docker_client is obtained via the line

    docker_client = docker.from_env()

I've tried running as

sudo -E ./myscript.py image

But this doesn't work either. I have a variant of this using a bash script and that works fine.

Output for the docker_client.login looks like

Looking for auth entry for 'ABCXYZ.dkr.ecr.us-west-2.amazonaws.com' Found 'ABCXYZ.dkr.ecr.us-west-2.amazonaws.com'

And if I dump the response, it looks like this.

{'password': 'PASSWORD HERE', 'email': None, 'username': 'AWS', 'serveraddress': 'ABCXYZ.dkr.ecr.us-west-2.amazonaws.com'}

Answer 1

Okay, I'm not quite sure if this is fully correct, however, based on some experimentation over the last few days, it does work. I also did open up an issue on the docker-py GitHub repo, but at least as of now, no one has chimed in.

https://github.com/docker/docker-py/issues/2256

In a nutshell, which I've listed in the link, here is what I came up with:

Okay, I ran some experiments the last few days. Had to deal with the 12 hour AWS ECR ticket so it took a little longer to do.

It does seem that there is an issue with docker-py.

Based on my findings, I can either use boto3 or run a sub-process calling the command line to aws ecr. However the only permutation that seems to work with the following steps.

1. use a sub-process to perform the docker login. This will result in the config.json file being updated (not sure if this has any relevance at all or not).
2. Create the docker client via docker_client = docker.from_env(). I have found doing this prior to the sub-process results in it not working properly (unless you have an already valid config.json
3. Then call docker_client.login(username=username, password=password, registry=registry_url)

Whether or not this is expected or not or if I'm doing something wrong, I don't know. This is what I've come up with as steps that work.