Menu
We have a Python client that connects to the Amazon S3 via a VPC endpoint. Our code uses boto and we are able to connect and download from S3.
After migration from boto to boto3, we noticed that the VPC endpoint connection no longer works. Below is a copy snippet that can reproduce the problem.
python -c "import boto3; s3 = boto3.resource('s3', aws_access_key_id='foo', aws_secret_access_key='bar'); s3.Bucket('some-bucket').download_file('hello-remote.txt', 'hello-local.txt')" got the below error:
Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Python27\lib\site-packages\boto3-1.4.0-py2.7.egg\boto3\s3\inject.py", line 163, in bucket_download_file ExtraArgs=ExtraArgs, Callback=Callback, Config=Config) File "C:\Python27\lib\site-packages\boto3-1.4.0-py2.7.egg\boto3\s3\inject.py", line 125, in download_file extra_args=ExtraArgs, callback=Callback) File "C:\Python27\lib\site-packages\boto3-1.4.0-py2.7.egg\boto3\s3\transfer.py ", line 269, in download_file future.result() File "build\bdist.win32\egg\s3transfer\futures.py", line 73, in result File "build\bdist.win32\egg\s3transfer\futures.py", line 233, in result botocore.vendored.requests.exceptions.ConnectionError: ('Connection aborted.', e rror(10060, 'A connection attempt failed because the connected party did not pro perly respond after a period of time, or established connection failed because c onnected host has failed to respond')) Does anyone know if boto3 support connection to S3 via VPC endpoint and/or was able to get it to work? We are using boto3-1.4.0.
556
In the navigation pane, under Virtual Private Cloud, choose Route Tables. Choose the route table associated with the VPC subnet that has Amazon S3 connectivity issues. Choose the Routes view. Confirm that there's a route to Amazon S3 using the gateway VPC endpoint.
Uploading a file to S3 Bucket using Boto3 The upload_file() method requires the following arguments: file_name – filename on the local filesystem. bucket_name – the name of the S3 bucket. object_name – the name of the uploaded file (usually equal to the file_name )
To troubleshoot this error, check the following: Verify the policy associated with the interface VPC endpoint and the S3 bucket. Verify that your network can connect to the S3 endpoints. Verify that your DNS can resolve to the S3 endpoints IP addresses.
Currently, two types of VPC endpoints can be used to connect to Amazon S3: interface VPC endpoint and gateway VPC endpoint. When you configure an interface VPC endpoint, an elastic network interface (ENI) with a private IP address is deployed in your subnet.
This is most likely a configuration error in your VPC endpoint policies. If your policies are correct, then Boto3 never knows exactly how it's able to reach the S3 location, it really is up to the policies to allow/forbid this type of traffic.
Here's a quick walkthrough of what you can do for troubleshooting: https://aws.amazon.com/premiumsupport/knowledge-center/connect-s3-vpc-endpoint/
Other relevant docs:
142
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
