Weird behaviour of boto inside docker

Question

I am running the following image:

docker run -it --rm python:3.4-slim  /bin/bash

inside it (as root), I installed:

root@dab02ca9c61d:~# apt-get update && apt-get install nano

and then boto

root@dab02ca9c61d:~# pip install boto

Then I wrote the following in /root/.boto:

[Credentials]
aws_access_key_id = some_long_string
aws_secret_access_key = another_bigger_string

[Boto]
debug = 2
num_retries = 10

(This file is the same that I have in my host machine)

The versions are:

root@dab02ca9c61d:~# python -V
Python 3.4.4
root@dab02ca9c61d:~# pip list
boto (2.38.0)
pip (7.1.2)
setuptools (18.2)
root@dab02ca9c61d:~# 

(Exactly the same that I have in my host/local machine)

If I fired the python console:

root@dab02ca9c61d:~# python
Python 3.4.4 (default, Jan  8 2016, 00:24:55) 
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto
>>> boto.connect_s3()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.4/site-packages/boto/__init__.py", line 141, in connect_s3
    return S3Connection(aws_access_key_id, aws_secret_access_key, **kwargs)
  File "/usr/local/lib/python3.4/site-packages/boto/s3/connection.py", line 190, in __init__
    validate_certs=validate_certs, profile_name=profile_name)
  File "/usr/local/lib/python3.4/site-packages/boto/connection.py", line 569, in __init__
    host, config, self.provider, self._required_auth_capability())
  File "/usr/local/lib/python3.4/site-packages/boto/auth.py", line 987, in get_auth_handler
    'Check your credentials' % (len(names), str(names)))
boto.exception.NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. ['HmacAuthV1Handler'] Check your credentials
>>> 

If I activate the logging:

>>> boto.set_stream_logger('boto')
>>> boto.connect_s3()                                                                                                                                                                                                
2016-01-11 01:53:23,577 boto [DEBUG]:Using access key found in config file.
2016-01-11 01:53:23,577 boto [DEBUG]:Using secret key found in config file.
2016-01-11 01:53:23,577 boto [DEBUG]:Retrieving credentials from metadata server.
--- Logging error ---
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/logging/__init__.py", line 978, in emit
    msg = self.format(record)
  File "/usr/local/lib/python3.4/logging/__init__.py", line 828, in format
    return fmt.format(record)
  File "/usr/local/lib/python3.4/logging/__init__.py", line 573, in format
    record.exc_text = self.formatException(record.exc_info)
  File "/usr/local/lib/python3.4/logging/__init__.py", line 523, in formatException
    traceback.print_exception(ei[0], ei[1], tb, None, sio)
  File "/usr/local/lib/python3.4/traceback.py", line 169, in print_exception
    for line in _format_exception_iter(etype, value, tb, limit, chain):
  File "/usr/local/lib/python3.4/traceback.py", line 146, in _format_exception_iter
    for value, tb in values:
  File "/usr/local/lib/python3.4/traceback.py", line 125, in _iter_chain
    context = exc.__context__
AttributeError: 'NoneType' object has no attribute '__context__'
Call stack:
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.4/site-packages/boto/__init__.py", line 141, in connect_s3
    return S3Connection(aws_access_key_id, aws_secret_access_key, **kwargs)
  File "/usr/local/lib/python3.4/site-packages/boto/s3/connection.py", line 190, in __init__
    validate_certs=validate_certs, profile_name=profile_name)
  File "/usr/local/lib/python3.4/site-packages/boto/connection.py", line 555, in __init__
    profile_name)
  File "/usr/local/lib/python3.4/site-packages/boto/provider.py", line 200, in __init__
    self.get_credentials(access_key, secret_key, security_token, profile_name)
  File "/usr/local/lib/python3.4/site-packages/boto/provider.py", line 376, in get_credentials
    self._populate_keys_from_metadata_server()
  File "/usr/local/lib/python3.4/site-packages/boto/provider.py", line 391, in _populate_keys_from_metadata_server
    data='meta-data/iam/security-credentials/')
  File "/usr/local/lib/python3.4/site-packages/boto/utils.py", line 394, in get_instance_metadata
    return _get_instance_metadata(metadata_url, num_retries=num_retries, timeout=timeout)
  File "/usr/local/lib/python3.4/site-packages/boto/utils.py", line 234, in _get_instance_metadata
    return LazyLoadMetadata(url, num_retries, timeout)
  File "/usr/local/lib/python3.4/site-packages/boto/utils.py", line 244, in __init__
    data = boto.utils.retry_url(self._url, num_retries=self._num_retries, timeout=self._timeout)
  File "/usr/local/lib/python3.4/site-packages/boto/utils.py", line 224, in retry_url
    boto.log.exception('Caught exception reading instance data')
Message: 'Caught exception reading instance data'
Arguments: ()
2016-01-11 01:53:24,582 boto [ERROR]:Unable to read instance data, giving up
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.4/site-packages/boto/__init__.py", line 141, in connect_s3
    return S3Connection(aws_access_key_id, aws_secret_access_key, **kwargs)
  File "/usr/local/lib/python3.4/site-packages/boto/s3/connection.py", line 190, in __init__
    validate_certs=validate_certs, profile_name=profile_name)
  File "/usr/local/lib/python3.4/site-packages/boto/connection.py", line 569, in __init__
    host, config, self.provider, self._required_auth_capability())
  File "/usr/local/lib/python3.4/site-packages/boto/auth.py", line 987, in get_auth_handler
    'Check your credentials' % (len(names), str(names)))
boto.exception.NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. ['HmacAuthV1Handler'] Check your credentials
>>> 

(This example runs in my local machine)

I already read Boto Credential Error with Python on Windows Boto: Dynamically get aws_access_key_id and aws_secret_access_key in Python code from config? and another questions. Nothing works.

After that, I review the source code of boto.pyami and If I execute line by line got nothing.

For example:

>>> boto.config.sections()
['Credentials', 'Boto']
>>> boto.config.get_value('Credentials', 'aws_access_key_id')
>>> 

Note that the last command returns None.

The boto.config use ConfigParser, if I use that, it works:

>>> import configparser
>>> config = configparser.ConfigParser()
>>> config.read('~/.boto')
[]
>>> config.read('/root.boto')                                         
['/root/.boto']
>>> config.get('Credentials', 'aws_access_key_id')
'some_long_string'
>>> 

(Note that if I use the relative path It didn't work)

Finally if I use:

>> boto.connect_s3('some_long_string', 'another_bigger_string')
2016-01-11 02:05:33,636 boto [DEBUG]:Using access key provided by client.
2016-01-11 02:05:33,636 boto [DEBUG]:Using secret key provided by client.
S3Connection:s3.amazonaws.com
>>> 

I did another tests:I instllaed awscli and it works without trouble, put the config file in /etc/boto.cfg and didn't work.

I already spent all my weekend in this, any ideas?

Answer 1

Since I can not comment I'll put this as answer but not entirely sure what the problem is with the docker but I was able to replicate the problem as well. Might want to make an issue on boto to see if someone there can help. However I was able to get it working by setting the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and then starting python. Don't know if that helps but just wanted to throw that out there since I spent some time looking at it.