Ansible troubleshooting - AWS Failed to import the required Python library (botocore or boto3)

Introduction

Today we’re going to talk about Ansible troubleshooting, specifically about the “Failed to import the required Python library (botocore or boto3)” message and enable Ansible For AWS.

This fatal error message happens when we are trying to execute some code against your AWS EC2 Infrastructure without the necessary Python libraries for the AWS.

These circumstances are usually related to the configuration of your Ansible Controller node and usually are not related to Ansible Playbook.

I’m Luca Berton and welcome to today’s episode of Ansible Pilot.

## Playbook

The best way of talking about Ansible troubleshooting is to jump in a live Playbook to show you practically the “Failed to import the required Python library (botocore or boto3)” and how to solve it!

In this Playbook, I’m going to reproduce the error and fix using the PIP, the Python Package Manager on a demo machine.

error execution

$ ansible-playbook ami_search.yml 
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit
localhost does not match 'all'
PLAY [AMI search] *********************************************************************************
TASK [search for AMI] ********************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to import the required Python library (botocore or boto3) on demo.example.com's Python /usr/bin/python3.8. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"}
PLAY RECAP ****************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

fix code

• python version

The first step is to determine your Python version (3.8 in this example):

$ ansible --version
ansible [core 2.12.2]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/devops/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.8/site-packages/ansible
  ansible collection location = /home/devops/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.8.12 (default, Sep 16 2021, 10:46:05) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)]
  jinja version = 2.10.3
  libyaml = True
[devops@demo aws]$ python --version
-bash: python: command not found
[devops@demo aws]$ python3 --version
Python 3.6.8
[devops@demo aws]$ python3.8 --version
Python 3.8.12
[devops@demo aws]$ whereis python3.8
python3: /usr/bin/python3.6 /usr/bin/python3.6m /usr/bin/python3 /usr/bin/python3.8 /usr/lib/python3.6 /usr/lib/python3.8 /usr/lib64/python3.6 /usr/lib64/python3.8 /usr/local/lib/python3.8 /usr/include/python3.6m /usr/include/python3.8 /usr/share/man/man1/python3.1.gz

• before Let’s use the pip3.8 command because we are running python 3.8. In other Linux distributions, you might need to specify different Python versions. For example Python 3.9 using pip3.9, pip3 or just pip.

[root@demo aws]# pip3.8 list | grep boto

• fixed with PIP

[root@demo aws]# pip3.8 install boto3
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3.8 install --user` instead.
Collecting boto3
  Downloading https://files.pythonhosted.org/packages/d4/c0/59513bab408fe6dd366f55c5e6ea4128daffe501fa8986749932219bdd59/boto3-1.24.27-py3-none-any.whl (132kB)
     |████████████████████████████████| 133kB 616kB/s 
Collecting jmespath<2.0.0,>=0.7.1
  Downloading https://files.pythonhosted.org/packages/31/b4/b9b800c45527aadd64d5b442f9b932b00648617eb5d63d2c7a6587b7cafc/jmespath-1.0.1-py3-none-any.whl
Collecting botocore<1.28.0,>=1.27.27
  Downloading https://files.pythonhosted.org/packages/20/7a/06940ad3f89e19ef065dd2600b14148673efe775da4fa61f39b7c63724a8/botocore-1.27.27-py3-none-any.whl (9.0MB)
     |████████████████████████████████| 9.0MB 2.5MB/s 
Collecting s3transfer<0.7.0,>=0.6.0
  Downloading https://files.pythonhosted.org/packages/5e/c6/af903b5fab3f9b5b1e883f49a770066314c6dcceb589cf938d48c89556c1/s3transfer-0.6.0-py3-none-any.whl (79kB)
     |████████████████████████████████| 81kB 3.2MB/s 
Collecting python-dateutil<3.0.0,>=2.1
  Downloading https://files.pythonhosted.org/packages/36/7a/87837f39d0296e723bb9b62bbb257d0355c7f6128853c78955f57342a56d/python_dateutil-2.8.2-py2.py3-none-any.whl (247kB)
     |████████████████████████████████| 256kB 4.9MB/s 
Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/local/lib/python3.8/site-packages (from botocore<1.28.0,>=1.27.27->boto3) (1.26.9)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.8/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.28.0,>=1.27.27->boto3) (1.12.0)
Installing collected packages: jmespath, python-dateutil, botocore, s3transfer, boto3
Successfully installed boto3-1.24.27 botocore-1.27.27 jmespath-1.0.1 python-dateutil-2.8.2 s3transfer-0.6.0

• after

[root@demo aws]# pip3.8 list | grep boto
boto3              1.24.27  
botocore           1.27.27  
[root@demo aws]#

fix execution

$ ansible-playbook ami_search.yml 
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit
localhost does not match 'all'
PLAY [AMI search] *********************************************************************************
TASK [search for existing AMI] ********************************************************************
ok: [localhost]
TASK [debug] **************************************************************************************
ok: [localhost] => {
    "ami_found": {
        "changed": false,
        "failed": false,
        "images": [
            {
                "architecture": "x86_64",
                "block_device_mappings": [
                    {
                        "device_name": "/dev/sda1",
                        "ebs": {
                            "delete_on_termination": true,
                            "encrypted": false,
                            "snapshot_id": "snap-03f2e24f30f580353",
                            "volume_size": 10,
                            "volume_type": "gp2"
                        }
                    }
                ],
                "creation_date": "2020-11-02T11:01:38.000Z",
                "deprecation_time": "2022-11-02T11:01:38.000Z",
                "description": "Provided by Red Hat, Inc.",
                "ena_support": true,
                "hypervisor": "xen",
                "image_id": "ami-096fda3c22c1c990a",
                "image_location": "309956199498/RHEL-8.3.0_HVM-20201031-x86_64-0-Hourly2-GP2",
                "image_type": "machine",
                "name": "RHEL-8.3.0_HVM-20201031-x86_64-0-Hourly2-GP2",
                "owner_id": "309956199498",
                "platform_details": "Red Hat Enterprise Linux",
                "public": true,
                "root_device_name": "/dev/sda1",
                "root_device_type": "ebs",
                "sriov_net_support": "simple",
                "state": "available",
                "tags": {},
                "usage_operation": "RunInstances:0010",
                "virtualization_type": "hvm"
            }
        ]
    }
}
PLAY RECAP ****************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

See also: Ansible troubleshooting - VMware Failed to Import PyVmomi

Conclusion

Now you know better how to troubleshoot the Ansible “Failed to import the required Python library (botocore or boto3)” message and move forward with your Ansible For AWS project.

Related Articles

• AWS automation with Ansible