Ansible aws_s3 module fails says Boto3 is missing when it is not

Question

This Play installs python3, pip3, boto3 and botocore, and tries to use aws_s3 module to download a file:

TASK [run yum update -y using yum module] 
**********************************************************************
ok: [ip-10-200-2-137.us-west-2.compute.internal]

TASK [Install python3 and pip3] *************************************************************************************************
changed: [ip-10-200-2-137.us-west-2.compute.internal]

TASK [Install boto3 and botocore with pip3 module] ******************************************************************************
changed: [ip-10-200-2-137.us-west-2.compute.internal]

TASK [Create a directory if it does not exist using file module] ****************************************************************
changed: [ip-10-200-2-137.us-west-2.compute.internal]

TASK [downlod file from s3 with aws_s3 module] **********************************************************************************
fatal: [ip-10-200-2-137.us-west-2.compute.internal]: FAILED! => 
{"changed": false, "msg": "Python modules \"botocore\" or \"boto3\" 
are missing, please install both"}

It fails because it says boto3 is missing, but it actually it is not:

From the Target host you can see that boto3 was installed:

[ec2-user@ip-10-200-2-137 ~]$ pip3 freeze
boto3==1.9.120
botocore==1.12.120
docutils==0.14
jmespath==0.9.4
python-dateutil==2.8.0
s3transfer==0.2.0
six==1.12.0
urllib3==1.24.1
[ec2-user@ip-10-200-2-137 ~]

This is the task that installed boto3:

- name: Install boto3 and botocore with pip3 module
    pip:
      name: 
      - boto3
      - botocore
      executable: pip-3.7

This is the task that fails:

- name: downlod file from s3 with aws_s3 module 
    aws_s3:
      bucket: mybucket
      object: mybucket/jre-8u201-linux-x64.tar.gz
      dest: /home/ec2-user/updater/jre-8u201-linux-x64.tar.gz
      mode: get   

The target host does have two versions of Python installed:

[ec2-user@ip-10-200-2-157 ~]$ which python
/usr/bin/python
[ec2-user@ip-10-200-2-157 ~]$ which python3
/usr/bin/python3

My config file looks like this:

[defaults]
private_key_file=/home/ec2-user/manual-builds/key.pem
ansible_python_interpreter=/usr/bin/python3

Is this a bug? I see some similar questions have been asked going back almost a year, but I see no solutions - thanks much for any help.

Answer 1

The problem was that my playbook had two tasks and Ansible was using the python2 interpreter for the first one AND the second one. The second task needed the python3 interpreter to work so I had to specifiy it at the task level:

- name: downlod file from s3 with aws_s3 module
  vars:
      ansible_python_interpreter: /usr/bin/python3    
  aws_s3:
      bucket: launch-data
      object: jre-8u201-linux-x64.tar.gz
      dest: /home/ec2-user/updater/jre-8u201-linux-x64.tar.gz
      mode: get 

Answer 2

Ansible use /usr/bin/python as default python interpreter. And you install AWS libraries for python3 only:

- name: Install boto3 and botocore with pip3 module
    pip:
      name: 
      - boto3
      - botocore
      executable: pip-3.7

You can install AWS libraries to python2 by using pip or install for both (python3 and python2) or you can define: ansible_python_interpreter=/usr/bin/python3 in your inventory file then you limit ansible execution to python3 only.