Directly support fedora packaging of ansible devtools projects

[ssbarnea]

Ansible DevTools team wants to directly support packaging for Fedora for all upstream projects developed and maintained by our team.

We want to ensure that at least one team member is listed as packager and co-maintainer so we can step-in to make quick changes when needed.

We do have upstream pipelines using packit that run on any incoming pull request, and we want to keep these pipelines green.

Our goal is to minimize the delay between an upstream release on https://pypi.org and the moment the new rpm becomes available in both Fedora current stable and rawhide.

List of rpms from https://packages.fedoraproject.org/ that we need to watch for:

• ansible-lint - Maintainers: pnemade and gotmax23
• python-ansible-compat Maintainers: gotmax23, chedi, kevin
• python-subprocess-tee Maintainer: chedi
• python-molecule - Maintainer: chedi
• ansible-navigator (not packaged yet)

Note: Most of maintainers can be reached via [email protected] email.

Actions

• ensure that ansible-compat package is updated asap, as this is currently preventing building of the other rpms, as they require latest version.
• get ssbarnea listed a co-maintainer
• package navigator (low priority)

[gotmax23]

It would be great to have more involvement from upstream ansible projects in Fedora packaging!

Some notes:

ansible-lint - Maintainers: pnemade and gotmax23

I'm not a co-maintainer of that package. I just submitted a PR for the last update.

Note: Most of maintainers can be reached via [email protected] email.

This is for the ansible package. Don't send emails there if you want to reach the python-molecule or ansible-lint maintainers.

List of rpms from https://packages.fedoraproject.org/ that we need to watch for:

python-molecule-podman and python-molecule-docker as well.

Actions

Before proceeding, you (or whoever else on the devtools team) need to get sponsored into the packager group. Then, the Fedora maintainers will have to agree to adding you as a co-maintainer. The documentation about the sponsorship process is a bit scattered :(. I'd take a look at https://docs.fedoraproject.org/en-US/fesco/Packager_sponsor_policy/. Next time there's an update to one of the devtools projects, feel free to ping me in #packaging:ansible.com for assistance if you'd like to submit a PR to update the Fedora package. There are a couple pitfalls.

[pnemade]

I will only speak for ansible-lint package here:
I have added this package in Fedora and maintaining it since then in EPEL and Fedora. The problem I see in maintaining this package is about dependencies, mostly jsonschema dependency.

For ansible-lint Fedora packaging as upstream started rapidly updating dependency versions with their latest upstream released versions, but the same is not available in Fedora, causes delay sometimes in updating ansible-lint package.
Also, I do not like building alpha/beta upstream releases in Fedora as for that package versioning schema changes.

The problematic dependency for ansible-lint I found is for package python-jsonschema. Though maintainers of python-jsonschema in Fedora are nice and always accepts my PR but for EPEL/RHEL releases this package is not getting updated.
The python-ansible-compat package maintainer chedi, asked RHEL maintainers of python-jsonschema to update it in RHEL9 but we got answer that there is no capacity internally in Red Hat to update it. Hence for EPEL9, we are stuck with ansible-lint-5.4.0-2.el9 build forever. For EPEL8, we need pyproject rpm macros and other new packages to build same latest ansible-lint in EPEL8.

If you check there was no update at all happened in Fedora for python-jsonschema from 3.2.0 to 4.6.0 version. I started pushing PR's from 4.6.0 version for python-jsonschema.

[ssbarnea]

AFAIK, we bump dependencies such ansible-compat and python-jsonschema only when needed but that is indeed often as we discover bugs in them. I think we could vendor it to reduce dependency on external packages, similar with how pip does with its own deps.

I can tell you that ansible lint 5.4 is of very little use as we no longer support it in any way and ansible ecosystem is moving fast. Especially for the linter we need to be able to update it as often as possible.

[gotmax23]

For EPEL8, we need pyproject rpm macros and other new packages to build same latest ansible-lint in EPEL8.

I don't think the EPEL 8 ansible-lint package works at all right now. It's packaged for Python 3.6 but ansible is packaged for Python 3.9. If you want to make this work for EPEL 8, you'd have to package a bunch of new dependencies for python39.

It's possible to create a stub setup.py and use %py3_build and %py3_install. You can add echo 'from setuptools import setup; setup()' > setup.py in %prep or add %global py_setup -c 'from setuptools import setup(); setup'. However this won't work for ansible-lint which uses PEP 621 that requires a newer setuptools. In addition to the other hurdles, you'd have to revert that commit and maintain a setup.cfg downstream. Reverting this commit upstream would help us out.

F36 and EPEL 9 do support the pyproject macros, but ansible-lint's usage of PEP 621 and the minimum python-jsonschema requirement also block updating ansible-lint in those releases.

I think we could vendor it to reduce dependency on external packages, similar with how pip does with its own deps.

I don't like this solution. Our Packaging Guidelines strongly discourage vendoring. Ideally, we could make this compatible with EL9's python-jsonschema version. If you'd like to add support for vendoring dependencies, please make it optional like ansible-core does. We might be able to justify vendoring this on EPEL branches, but we'd have to add bundled() Provides and make sure licenses are properly handled.