Error loading shared library libresolv.so.2 in Python Alpine Docker

Question

I am using Alpine Linux as Docker image for my Python app in CI/CD (Jenkins). This is for AWS Lambda, basically the only noteworthy requirement is aws-psycopg2 for accessing Postgres. Locally everything works, it deploys nicely with Serverless and works on AWS, the only problem is on Jenkins. I am getting an error there:

16:24:55  ImportError: Error loading shared library libresolv.so.2: No such file or directory (needed by /usr/local/lib/python3.9/site-packages/psycopg2/../psycopg2_binary.libs/libgssapi_krb5-497db0c6.so.2.2)

I am NOT using Oracle in any way, like e.g. in this question.

My jenkinsfile.groovy:

def testInstallStage = {
  stage('Install') {
    sh 'apk add python3-dev gcc libc-dev musl-dev openblas gfortran build-base postgresql-libs postgresql-dev libffi-dev cargo'
    sh 'pip install --upgrade pip && pip install poetry'
    sh 'poetry export --without-hashes --dev -f requirements.txt -o requirements.txt --with-credentials'
    sh 'pip install -r requirements.txt'

    sh 'pip install awscli'
    sh 'aws configure set aws_access_key_id default_access_key'
    sh 'aws configure set aws_secret_access_key default_secret_key'
    sh 'aws configure set default.region eu-west-1'

    sh 'cp config/jenkins.json config.json'
  }
}

def testStage = {
  stage('Tests') {
    sh 'pip install nose && nosetests'
  }
}

new pl.ruby.PipelineRunner(this, notificationChannel).run() {
  pythonStack([image: '3.9.13-alpine3.16'], {
    container('python') {
      testInstallStage.call()
      testStage.call()
    }
  })
}

The whole line with apk add is from me trying to fix this. I tried installing many different dependencies. What should I install to get this working? I know that this is some library missing on Alpine, since on Ubuntu 20.04 and on AWS Lambda runtime it works ok.

Answer 1

The library libresolv.so.2 is part of the system C library. Alpine-based images use a different version of the C library (musl libc) which is much smaller, but also occasionally runs into compatibility issues like this.

The easiest way around this is to switch to a Debian- or Ubuntu-based image based on the more standard GNU libc. Since you're using an unmodified image here you'd just need to remove the "Alpine" reference in the image tag

pythonStack([image: '3.9.13'], ...)

In a custom Dockerfile you might also need to change the package manager invocations if you need to install any OS-level packages, since Debian/Ubuntu and Alpine have different packaging formats, tools, and names. The first line of your testInstallStage would also be affected by this.

The typical reason to use an Alpine-based image is to save space. The actual space savings here are fairly small (tens of megabytes) and once you start installing C header files the images quickly get much larger than this. You're probably not going to notice a practical difference in performance or image size making this change.