SSLError using boto

Question

We are using a proxy + profile when using the aws s3 commands to browse our buckets in CLI.

export HTTPS_PROXY=https://ourproxyhost.com:3128
aws s3 ls s3://our_bucket/.../ --profile dev

And we can work with our buckets and objects fine.

Because I need to write Python code for this, I translated this using boto3:

# python 2.7.12
import boto3                        # v1.5.18
from botocore.config import Config  # v1.8.32

s3 = boto3.Session(profile_name='dev').resource('s3', config=Config(proxies={'https': 'ourproxyhost.com:3128'})).meta.client
obj = s3.get_object(Bucket='our_bucket', Key='dir1/dir2/.../file')

What I get is this:

botocore.vendored.requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

Why is this working in CLI, but not in Python?

Answer 1

 botocore.vendored.requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

The error above in most cases it's usually related to the CA bundle being used for S3 connections.

Possible Resolution Steps:

1. Turn off SSL certification validation :

s3 = boto3.client('s3', verify=False)

As mentioned in this boto3 documentation, this option turns off validation of SSL certificates but SSL protocol will still be used (unless use_ssl is False) for communication.

2. Check if you have AWS_CA_BUNDLE env var set?:

echo $AWS_CA_BUNDLE

or

export | grep AWS_CA_BUNDLE

3. Check if you have certifi installed in your python env?:

pip list | grep certifi

Depending on the output of the above command, you could be using a version of certifi (which is not a dependency of boto3) that has a broken certificate validation when communicating with s3 endpoints.

You will need to upgrade your OpenSSL version or pin certifi to a stable version as shown below :

sudo pip uninstall certifi
sudo pip install certifi==2015.04.28

Hope this helps!