Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

community.crypto.openssl_privatekey module also needs 'cffi' #804

Open
villapx-path opened this issue Oct 3, 2024 · 5 comments
Open

community.crypto.openssl_privatekey module also needs 'cffi' #804

villapx-path opened this issue Oct 3, 2024 · 5 comments
Labels
question Further information is requested

Comments

@villapx-path
Copy link

SUMMARY

The community.crypto.openssl_privatekey module also requires the cffi package, however the documentation only lists cryptography as a dependency.

ISSUE TYPE
  • Documentation Report
COMPONENT NAME

community.crypto.openssl_privatekey

ANSIBLE VERSION
ansible [core 2.17.4]
  config file = /home/jvilla/projects/path-production-ansible/ansible.cfg
  configured module search path = ['/home/jvilla/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/jvilla/.local/pipx/venvs/ansible-core/lib/python3.10/site-packages/ansible
  ansible collection location = /home/jvilla/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/jvilla/.local/bin/ansible
  python version = 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] (/home/jvilla/.local/pipx/venvs/ansible-core/bin/python)
  jinja version = 3.1.4
  libyaml = True
@villapx-path
Copy link
Author

villapx-path commented Oct 3, 2024 •
edited
Loading

Some more details:

I am calling the community.crypto.openssl_privatekey module, and it fails because it is missing the "cffi backend" module.

Here is the -vvv output of this failing task:

ansible-playbook -vvv output 
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: jvilla
<localhost> EXEC /bin/sh -c 'echo ~jvilla && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/jvilla/.ansible/tmp `"&& mkdir "` echo /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243 `" && echo ansible-tmp-1727964631.7938662-12315-25707778985243="` echo /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243 `" ) && sleep 0'
Using module file /home/jvilla/.ansible/collections/ansible_collections/community/crypto/plugins/modules/openssl_privatekey.py
<localhost> PUT /home/jvilla/.ansible/tmp/ansible-local-121319ffj6jk5/tmp8rv9m_a5 TO /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/ /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python3.11 /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py", line 107, in <module>
    _ansiballz_main()
  File "/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible_collections.community.crypto.plugins.modules.openssl_privatekey', init_globals=dict(_module_fqn='ansible_collections.community.crypto.plugins.modules.openssl_privatekey', _modlib_path=modlib_path),
  File "<frozen runpy>", line 226, in run_module
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/modules/openssl_privatekey.py", line 301, in <module>
  File "/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/modules/openssl_privatekey.py", line 281, in main
  File "/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/privatekey.py", line 498, in select_backend
  File "/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/privatekey.py", line 301, in __init__
  File "/usr/lib/python3/dist-packages/cryptography/hazmat/backends/__init__.py", line 14, in default_backend
    from cryptography.hazmat.backends.openssl.backend import backend
  File "/usr/lib/python3/dist-packages/cryptography/hazmat/backends/openssl/__init__.py", line 6, in <module>
    from cryptography.hazmat.backends.openssl.backend import backend
  File "/usr/lib/python3/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 114, in <module>
    from cryptography.hazmat.bindings.openssl import binding
  File "/usr/lib/python3/dist-packages/cryptography/hazmat/bindings/openssl/binding.py", line 14, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ModuleNotFoundError: No module named '_cffi_backend'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py\", line 107, in <module>\n    _ansiballz_main()\n  File \"/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py\", line 99, in _ansiballz_main\n    in
voke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/jvilla/.ansible/tmp/ansible-tmp-1727964631.7938662-12315-25707778985243/AnsiballZ_openssl_privatekey.py\", line 47, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.community.crypto.plugins.modules.openssl_privatekey', init_globals=dict(_module_fqn='ansible_collections.community.crypto.plugins
.modules.openssl_privatekey', _modlib_path=modlib_path),\n  File \"<frozen runpy>\", line 226, in run_module\n  File \"<frozen runpy>\", line 98, in _run_module_code\n  File \"<frozen runpy>\", line 88, in _run_code\n  File \"/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypt
o/plugins/modules/openssl_privatekey.py\", line 301, in <module>\n  File \"/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/modules/openssl_privatekey.py\", line 281, in main\n  File \"/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_commun
ity.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/privatekey.py\", line 498, in select_backend\n  File \"/tmp/ansible_community.crypto.openssl_privatekey_payload_lu_qg4ot/ansible_community.crypto.openssl_privatekey_payload.zip/ansible_collections/community/crypto/plugins/module_utils/crypto/module_backends/pr
ivatekey.py\", line 301, in __init__\n  File \"/usr/lib/python3/dist-packages/cryptography/hazmat/backends/__init__.py\", line 14, in default_backend\n    from cryptography.hazmat.backends.openssl.backend import backend\n  File \"/usr/lib/python3/dist-packages/cryptography/hazmat/backends/openssl/__init__.py\", line 6, in <module>\n    from cryptography.hazmat.backends.openssl.bac
kend import backend\n  File \"/usr/lib/python3/dist-packages/cryptography/hazmat/backends/openssl/backend.py\", line 114, in <module>\n    from cryptography.hazmat.bindings.openssl import binding\n  File \"/usr/lib/python3/dist-packages/cryptography/hazmat/bindings/openssl/binding.py\", line 14, in <module>\n    from cryptography.hazmat.bindings._openssl import ffi, lib\nModuleNot
FoundError: No module named '_cffi_backend'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

Per that output, it is executing /usr/bin/python3.11 on the host, which at the time of this error had the following packages installed:

pip list, when the error occurred 
$ /usr/bin/python3.11 -m pip list
Package             Version
------------------- -------------
argcomplete         1.8.1
attrs               21.2.0
Automat             20.2.0
Babel               2.8.0
bcrypt              3.2.0
blinker             1.4
certifi             2020.6.20
chardet             4.0.0
click               8.0.3
cloud-init          24.3.1
colorama            0.4.4
command-not-found   0.3
configobj           5.0.6
constantly          15.1.0
cryptography        3.4.8
dbus-python         1.2.18
distro              1.7.0
distro-info         1.1+ubuntu0.2
httplib2            0.20.2
hyperlink           21.0.0
idna                3.3
importlib-metadata  4.6.4
incremental         21.3.0
jeepney             0.7.1
Jinja2              3.0.3
jsonpatch           1.32
jsonpointer         2.0
jsonschema          3.2.0
keyring             23.5.0
launchpadlib        1.10.16
lazr.restfulclient  0.14.4
lazr.uri            1.0.6
livereload          2.6.3
Markdown            3.3.6
MarkupSafe          2.0.1
mkdocs              1.1.2
more-itertools      8.10.0
netifaces           0.11.0
oauthlib            3.2.0
packaging           21.3
pip                 22.0.2
pipx                1.0.0
psutil              5.9.0
pyasn1              0.4.8
pyasn1-modules      0.2.1
pycurl              7.44.1
Pygments            2.11.2
PyGObject           3.42.1
PyHamcrest          2.0.2
pyinotify           0.9.6
PyJWT               2.3.0
pyOpenSSL           21.0.0
pyparsing           2.4.7
pyrsistent          0.18.1
pyserial            3.5
python-apt          2.4.0+ubuntu4
pytz                2022.1
PyYAML              5.4.1
requests            2.25.1
SecretStorage       3.3.1
service-identity    18.1.0
setuptools          59.6.0
six                 1.16.0
systemd-python      234
tornado             6.1
Twisted             22.1.0
ubuntu-pro-client   8001
ufw                 0.36.1
unattended-upgrades 0.1
urllib3             1.26.5
userpath            1.8.0
wadllib             1.3.6
wheel               0.37.1
zipp                1.0.0
zope.interface      5.4.0

After installing cffi (output below), the task succeeded:

pip install cffi, and re-run playbook 
[10:14] jvilla@PATH-HPWNJX3:~/projects/my-playbook $ python3.11 -m pip install cffi
Defaulting to user installation because normal site-packages is not writeable
Collecting cffi
  Downloading cffi-1.17.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (467 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 467.2/467.2 KB 4.5 MB/s eta 0:00:00
Collecting pycparser
  Downloading pycparser-2.22-py3-none-any.whl (117 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 KB 26.9 MB/s eta 0:00:00
Installing collected packages: pycparser, cffi
Successfully installed cffi-1.17.1 pycparser-2.22
...
...
...
...
[10:14] jvilla@PATH-HPWNJX3:~/projects/my-playbook $ ansible-playbook -vvv generate_vpn_certs.yml
ansible-playbook [core 2.17.4]
  config file = /home/jvilla/projects/my-playbook/ansible.cfg
  configured module search path = ['/home/jvilla/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/jvilla/.local/pipx/venvs/ansible-core/lib/python3.10/site-packages/ansible
  ansible collection location = /home/jvilla/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/jvilla/.local/bin/ansible-playbook
  python version = 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] (/home/jvilla/.local/pipx/venvs/ansible-core/bin/python)
  jinja version = 3.1.4
  libyaml = True
...
...
... lots of -vvv output and normal task output ...
...
...
TASK [vpn_config_generate : Generate an OpenSSL private key with the default values (4096 bits, RSA)] ***
task path: /home/jvilla/projects/my-playbook/roles/vpn_config_generate/tasks/main.yml:33
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: jvilla
<localhost> EXEC /bin/sh -c 'echo ~jvilla && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/jvilla/.ansible/tmp `"&& mkdir "` echo /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826 `" && echo ansible-tmp-1727964878.9725957-12642-118435769711826="` echo /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826 `" ) && sleep 0'
Using module file /home/jvilla/.ansible/collections/ansible_collections/community/crypto/plugins/modules/openssl_privatekey.py
<localhost> PUT /home/jvilla/.ansible/tmp/ansible-local-124581t36wg16/tmp1ziz87xo TO /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826/AnsiballZ_openssl_privatekey.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826/ /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826/AnsiballZ_openssl_privatekey.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python3.11 /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826/AnsiballZ_openssl_privatekey.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/jvilla/.ansible/tmp/ansible-tmp-1727964878.9725957-12642-118435769711826/ > /dev/null 2>&1 && sleep 0'
changed: [localhost] => {
    "changed": true,
    "diff": {
        "after": {
            "can_parse_key": true,
            "key_is_consistent": null,
            "public_data": {
                <<<redacted>>>
            },
            "public_key": "-----BEGIN PUBLIC KEY-----\n<<<redacted>>>\n-----END PUBLIC KEY-----\n",
            "public_key_fingerprints": {
                "sha256": <<<redacted>>>
            },
            "type": "RSA"
        },
        "before": {}
    },
    "filename": "./ssl_tmp/client.key",
    "fingerprint": {
        <<<redacted>>>
    },
    "invocation": {
        "module_args": {
            "attributes": null,
            "backup": false,
            "cipher": "auto",
            "curve": null,
            "force": false,
            "format": "auto_ignore",
            "format_mismatch": "regenerate",
            "group": null,
            "mode": "0600",
            "owner": null,
            "passphrase": null,
            "path": "./ssl_tmp/client.key",
            "regenerate": "full_idempotence",
            "return_content": false,
            "select_crypto_backend": "auto",
            "selevel": null,
            "serole": null,
            "setype": null,
            "seuser": null,
            "size": 4096,
            "state": "present",
            "type": "RSA",
            "unsafe_writes": false
        }
    },
    "size": 4096,
    "type": "RSA"
}
Read vars_file 'openvpn_vaulted.yml'
Read vars_file 'terraform_vars.yml'

@felixfontein
Copy link
Contributor

The module only needs cryptography. cffi is a transitive dependency needed by cryptography: https://github.com/pyca/cryptography/blob/main/pyproject.toml#L52

@felixfontein felixfontein added the question Further information is requested label Oct 3, 2024
@villapx-path
Copy link
Author

@felixfontein Thanks for the response!

What do you think then is the "proper" course of action for me? This has already been filed as a bug with the cryptography project 7 years ago, and they said that it is explicitly configured to not list cffi as a dependency when installing cryptography from PyPi.

Is the requirement basically, "ensure you have cryptography installed on the target system, and also cffi if you installed cryptography via PyPi"?

@felixfontein
Copy link
Contributor

The issue you are linking is about PyPy (another Python engine), not PyPI (the Python package index). If you install cryptography via pip (or any other proper resolver) for CPython (or in fact any other Python engine than PyPy), cffi should also be installed since cffi is a dependency of cryptography.

(Your Python from ansible --version doesn't look like PyPy.)

If you have cryptography installed but not cffi, then your environment is broken.

@villapx-path
Copy link
Author

villapx-path commented Oct 3, 2024 •
edited
Loading

Gotcha lol, thanks for correcting my misunderstanding on the PyPy vs. PyPi 😄

then your environment is broken

This is a pretty standard install of the python3.11 apt package on Ubuntu 22.04 in WSL. Although, Ubuntu 22.04's default Python version (i.e. /usr/bin/python3) is Python 3.10, and so perhaps that is what was causing the issue?

Some details:

[18:04] jvilla@PATH-HPWNJX3:~ $ cat /etc/os-release 
PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
[17:04] jvilla@PATH-HPWNJX3:~ $ 
[17:04] jvilla@PATH-HPWNJX3:~ $ 
[17:04] jvilla@PATH-HPWNJX3:~ $ 
[17:04] jvilla@PATH-HPWNJX3:~ $ dpkg -l python3.11 python3-cryptography python3-cffi-backend
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                       Version            Architecture Description
+++-==========================-==================-============-=======================================================================
ii  python3-cffi-backend:amd64 1.15.0-1build2     amd64        Foreign Function Interface for Python 3 calling C code - runtime
ii  python3-cryptography       3.4.8-1ubuntu2.2   amd64        Python library exposing cryptographic recipes and primitives (Python 3)
ii  python3.11                 3.11.0~rc1-1~22.04 amd64        Interactive high-level object-oriented language (version 3.11)
[17:04] jvilla@PATH-HPWNJX3:~ $ 
[17:04] jvilla@PATH-HPWNJX3:~ $ 
[17:04] jvilla@PATH-HPWNJX3:~ $ dpkg -L python3-cffi-backend 
/.
/usr
/usr/lib
/usr/lib/python3
/usr/lib/python3/dist-packages
/usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so
/usr/share
/usr/share/doc
/usr/share/doc/python3-cffi-backend
/usr/share/doc/python3-cffi-backend/changelog.Debian.gz
/usr/share/doc/python3-cffi-backend/copyright

Not being quite this level of Python expert, the _cffi_backend.cpython-310-x86_64-linux-gnu.so seems somewhat suspicious to me, in that Python 3.11 maybe isn't able to dynamically load that shared library that's (maybe) only compatible with Python 3.10?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@felixfontein @villapx-path