cryptography python module not found when using community.crypto

[kwikmr2]

SUMMARY

Built AWX EE from https://github.com/Frewx/awx-ee-builder.git.
Defined "quay.io/ansible/awx-ee:latest" in execution-environment.yml for base_image.
Included "cryptography" in requirements.txt.
Included "community.general" (latest) and "community.crypto" (latest) in requirements.yml.

When executing a simple task via AWX, the following error occurs:
"Cannot detect any of the required Python libraries cryptography (>= 1.6)"

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.crypto.x509_certificate_info

ANSIBLE VERSION

ansible [core 2.15.12]
  config file = None
  configured module search path = [‘/runner/.ansible/plugins/modules’, ‘/usr/share/ansible/plugins/modules’]
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.18 (main, Jan 24 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

COLLECTION VERSION

# /usr/local/lib/python3.9/site-packages/ansible_collections
Collection       Version
---------------- -------
community.crypto 2.16.1
# /usr/share/ansible/collections/ansible_collections
Collection       Version
---------------- -------
community.crypto 2.20.0

CONFIGURATION

CONFIG_FILE() = None

OS / ENVIRONMENT

quay.io/ansible/awx-ee:latest based on CentOS Stream release 9

STEPS TO REPRODUCE

Deploy EE based on the settings posted above. Setup EE in AWX (24.3.1) and execute simple playbook against target host.

---
  - name: Get information on generated certificate
    community.crypto.x509_certificate_info:
      path: /data/path/test-ca.crt
    register: result

  - name: Dump certificate information
    ansible.builtin.debug:
      var: result

EXPECTED RESULTS

That the information about a certificate would return

ACTUAL RESULTS

TASK [Get information on generated certificate] ********************************
task path: /runner/project/tasks/ssl-main.yml:2
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'echo ~ansible_svc && sleep 0'"'"''
<192.168.57.104> (0, b'/home/ansible_svc\\n', b"Warning: Permanently added '192.168.57.104' (ECDSA) to the list of known hosts.\\r\\n<redacted>.\\n")
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/ansible_svc/.ansible/tmp `"&& mkdir "` echo /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843 `" && echo ansible-tmp-1717007346.3548055-27-128022922135843="` echo /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843 `" ) && sleep 0'"'"''
<192.168.57.104> (0, b'ansible-tmp-1717007346.3548055-27-128022922135843=/home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843\\n', b'')
<labymrepo01> Attempting python interpreter discovery
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'echo PLATFORM; uname; echo FOUND; command -v '"'"'"'"'"'"'"'"'python3.12'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.11'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.10'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.9'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.8'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.6'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/bin/python3'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/libexec/platform-python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python2.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/bin/python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python'"'"'"'"'"'"'"'"'; echo ENDFOUND && sleep 0'"'"''
<192.168.57.104> (0, b'PLATFORM\\nLinux\\nFOUND\\n/usr/libexec/platform-python\\n/usr/bin/python2.7\\n/usr/bin/python\\n/usr/bin/python\\nENDFOUND\\n', b'')
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'/usr/libexec/platform-python && sleep 0'"'"''
<192.168.57.104> (0, b'{"osrelease_content": "NAME=\\\\"CentOS Linux\\\\"\\\\nVERSION=\\\\"7 (Core)\\\\"\\\\nID=\\\\"centos\\\\"\\\\nID_LIKE=\\\\"rhel fedora\\\\"\\\\nVERSION_ID=\\\\"7\\\\"\\\\nPRETTY_NAME=\\\\"CentOS Linux 7 (Core)\\\\"\\\\nANSI_COLOR=\\\\"0;31\\\\"\\\\nCPE_NAME=\\\\"cpe:/o:centos:centos:7\\\\"\\\\nHOME_URL=\\\\"https://www.centos.org/\\\\"\\\\nBUG_REPORT_URL=\\\\"https://bugs.centos.org/\\\\"\\\\n\\\\nCENTOS_MANTISBT_PROJECT=\\\\"CentOS-7\\\\"\\\\nCENTOS_MANTISBT_PROJECT_VERSION=\\\\"7\\\\"\\\\nREDHAT_SUPPORT_PRODUCT=\\\\"centos\\\\"\\\\nREDHAT_SUPPORT_PRODUCT_VERSION=\\\\"7\\\\"\\\\n\\\\n", "platform_dist_result": ["centos", "7.9.2009", "Core"]}\\n', b'')
Using module file /usr/share/ansible/collections/ansible_collections/community/crypto/plugins/modules/x509_certificate_info.py
<192.168.57.104> PUT /runner/.ansible/tmp/ansible-local-22yu_u7i7f/tmp5su6kh3e TO /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/AnsiballZ_x509_certificate_info.py
<192.168.57.104> SSH: EXEC sshpass -d12 sftp -o BatchMode=no -b - -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' '[192.168.57.104]'
<192.168.57.104> (0, b'sftp> put /runner/.ansible/tmp/ansible-local-22yu_u7i7f/tmp5su6kh3e /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/AnsiballZ_x509_certificate_info.py\\n', b'')
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'chmod u+x /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/ /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/AnsiballZ_x509_certificate_info.py && sleep 0'"'"''
<192.168.57.104> (0, b'', b'')
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' -tt 192.168.57.104 '/bin/sh -c '"'"'sudo -H -S -p "[sudo via ansible, key=etjodrossjnhiiejjjipkloplcelzydr] password:" -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-etjodrossjnhiiejjjipkloplcelzydr ; /usr/bin/python /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/AnsiballZ_x509_certificate_info.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<192.168.57.104> (1, b'\\r\\n{"msg": "Cannot detect any of the required Python libraries cryptography (>= 1.6)", "failed": true, "invocation": {"module_args": {"content": null, "select_crypto_backend": "auto", "name_encoding": "ignore", "valid_at": null, "path": "/data/path/test-ca.crt"}}}\\r\\n', b'Shared connection to 192.168.57.104 closed.\\r\\n')
<192.168.57.104> Failed to connect to the host via ssh: Shared connection to 192.168.57.104 closed.
<192.168.57.104> ESTABLISH SSH CONNECTION FOR USER: ansible_svc
<192.168.57.104> SSH: EXEC sshpass -d12 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'User="ansible_svc"' -o ConnectTimeout=10 -o 'ControlPath="/runner/cp/30094d2be6"' 192.168.57.104 '/bin/sh -c '"'"'rm -f -r /home/ansible_svc/.ansible/tmp/ansible-tmp-1717007346.3548055-27-128022922135843/ > /dev/null 2>&1 && sleep 0'"'"''
<192.168.57.104> (0, b'', b'')
fatal: [labymrepo01]: FAILED! => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "invocation": {
        "module_args": {
            "content": null,
            "name_encoding": "ignore",
            "path": "/data/path/test-ca.crt",
            "select_crypto_backend": "auto",
            "valid_at": null
        }
    },
    "msg": "Cannot detect any of the required Python libraries cryptography (>= 1.6)"
}

[felixfontein]

You need to install the cryptography dependency for the Python where the module is executed. If you execute the module on a target node, you have to make sure it's also installed there. Ansible, AWX, the collection, requirements.txt, ... won't do that automatically for you.

[kwikmr2]

I just noticed in the debug output that Python2.7 is being used:
"04> (0, b'PLATFORM\nLinux\nFOUND\n/usr/libexec/platform-python\n/usr/bin/python2.7\n/usr/bin/python\n/usr/bin/python\nENDFOUND\n', b'')"

How is this possible when the EE is CentOS Stream 9 with Python3.9 installed? The whole purpose of the EE is to execute within that environment.

[felixfontein]

If you ask Ansible to run the task on a remote target, then it won't be run inside the EE, but on the remote target.

If you want to run the module in the EE, you have to use hosts: localhost or delegate_to: localhost.

[kwikmr2]

I made the adjustments and now it fails because the path to the certificate is not on the localhost (the EE)...this is seems to be a paradox.

---
  - name: Get information on generated certificate
    community.crypto.x509_certificate_info:
      path: /data/path/test-ca.crt
    register: result
    delegate_to: localhost

Error:

The full traceback is:
  File "/tmp/ansible_community.crypto.x509_certificate_info_payload_lcdbcu61/ansible_community.crypto.x509_certificate_info_payload.zip/ansible_collections/community/crypto/plugins/modules/x509_certificate_info.py", line 444, in main
fatal: [labymrepo01 -> localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "content": null,
            "name_encoding": "ignore",
            "path": "/data/path/test-ca.crt",
            "select_crypto_backend": "auto",
            "valid_at": null
        }
    },
    "msg": "Error while reading certificate file from disk: [Errno 2] No such file or directory: '/data/path/test-ca.crt'"
}

[felixfontein]

Well, if you want to operate on files on the remote, you either first have to fetch them to the controller, or you have to run the module on the target - but then you have to make sure that the module's requirements are available.

[kwikmr2]

Okay, then assuming it is not feasible to install the cryptography python module on every single remote host in the inventory AND using delegate_to: localhost breaks since the certificate to be inspected is not on the EE...that would make this in a way only usable with CLI Ansible and not AWX/Tower.

[felixfontein]

I don't see what the difference between CLI Ansible and AWX/Tower is. In both cases, installing on the controller is easier than installing on all remotes, and you can determine yourself whether to run on the controller or on the targets.

[mstyne]

I'm encountering an issue similar to this on Rocky Linux 9 (works fine on Ubuntu 22.04 and Debian 12) where cryptography is not discovered when Ansible is run in a Python venv.

ansible --version reports the correct Python interpreter path.

# ansible --version
ansible [core 2.15.12]
  config file = None
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /root/.local/venv/lib64/python3.9/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /root/.local/venv/bin/ansible
  python version = 3.9.18 (main, Jul  3 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/root/.local/venv/bin/python3)
  jinja version = 3.1.4
  libyaml = True

Debug output from:

community.general.python_requirements_info:
  dependencies: cryptography

    "changed": false,
    "invocation": {
        "module_args": {
            "dependencies": [
                "cryptography"
            ]
        }
    },
    "mismatched": {},
    "not_found": [
        "cryptography"
    ],
    "python": "/usr/bin/python3",
    "python_system_path": [
        "/tmp/ansible_community.general.python_requirements_info_payload_wo1d5_v7/ansible_community.general.python_requirements_info_payload.zip",
        "/usr/lib64/python39.zip",
        "/usr/lib64/python3.9",
        "/usr/lib64/python3.9/lib-dynload",
        "/root/.local/lib/python3.9/site-packages",
        "/usr/lib64/python3.9/site-packages",
        "/usr/lib/python3.9/site-packages"
    ],
    "python_version": "3.9.18 (main, Jul  3 2024, 00:00:00) \n[GCC 11.4.1 20231218 (Red Hat 11.4.1-3)]",
    "python_version_info": {
        "major": 3,
        "micro": 18,
        "minor": 9,
        "releaselevel": "final",
        "serial": 0
    },
    "valid": {}
}

...indicates cryptography cannot be found, and the python interpreter uses the system python binary, and python_system_path shows the venv nowhere to be found. Yes, the venv has been activated prior to invoking Ansible.

(venv) # pip3 list | grep cryptography
cryptography        43.0.0

Overriding ansible_python_interpreter to use the Python binary from the venv resolves the issue, so it's possible this is an "Ansible on CentOS (and friends)" issue, rather than a community.crypto issue. I don't have a great solution for this, but I'm putting this out into the world in case someone else finds their way down this sad path.

[felixfontein]

Overriding ansible_python_interpreter to use the Python binary from the venv resolves the issue, so it's possible this is an "Ansible on CentOS (and friends)" issue

Yes, that's exactly it. You need to point ansible_python_interpreter to the venv, otherwise Ansible won't use it to run modules when using connection: local.

[mstyne]

Overriding ansible_python_interpreter to use the Python binary from the venv resolves the issue, so it's possible this is an "Ansible on CentOS (and friends)" issue

Yes, that's exactly it. You need to point ansible_python_interpreter to the venv, otherwise Ansible won't use it to run modules when using connection: local.

I don't encounter this additional configuration requirement with Ubuntu or Debian, but I'll have to double check to verify. If that output is interesting, I can post it here.

[felixfontein]

You probably already have cryptography installed on system level on Debian and Ubuntu, and thus don't notice that it does not use the cryptography you explicitly installed but the system one.