pyverbs: Align code to support Cython 3.1.0 format

[YonatanNachum]

In Cython 3.1.0 named cpdef enums no longer copy their item names into the global module namespace. This means each named enum must be imported explicitly to be used in a file or else an import exception will be raised. Align pyverbs code to explicitly import enums so it will be able to execute on relatively new distributions that cython version in them is newer than 3.1.0: Ubuntu 22.04/24.04, Debian 12, RHEL 9, Rocky Linux 9 and AL2023.

This change was also tested on cython 3.0.12 to verify it works on older versions.

Link: https://cython.readthedocs.io/en/latest/src/changes.html#id10

[EdwardSro]

I understand the motivation, but the resulting code is cumbersome, especially when these enums are used frequently.
Would it be possible to provide shorter aliases or import patterns for common enums, at least in the test code, to improve readability?

[YonatanNachum]

@EdwardSro, I agree its indeed cumbersome but Cython changed their behavior from C like enums to python like enums so the code can't use the existing c-like implementation which is much more convenient where all enums are just exposed without the enum prefix.

I couldn't really find a reasonable way around it so indeed it became a huge change.

Would it be possible to provide shorter aliases or import patterns for common enums, at least in the test code, to improve readability?

The only option I see here is doing something like from pyverbs.libibverbs_enums import ibv_access_flags as x for example and than we can fetch values with x.IBV_ACCESS_LOCAL_WRITE. I don't find this approach really helpful and can be less readable in my opinion. What do you think ?

[EdwardSro]

The only option I see here is doing something like from pyverbs.libibverbs_enums import ibv_access_flags as x for example and than we can fetch values with x.IBV_ACCESS_LOCAL_WRITE. I don't find this approach really helpful and can be less readable in my opinion. What do you think ?

Maybe we can still globalize them using something like "globals().update(MyEnum.members)"?
Please see this discussion in Cython GitHub: cython/cython#4571
OG PR: cython/cython#6382

I believe we can avoid this significant change, which will affect external Pyverbs users as well.

[YonatanNachum]

@EdwardSro I did consider that option, but it feels more like a workaround than a proper solution. It essentially "hacks" Python’s global namespace to reintroduce the enums. While the change is smaller, it maintains a behavior that neither Python nor Cython officially supports—and there’s always the risk that this workaround could be removed in the future.

I’m open to changing it if you strongly prefer that approach, but in my view, it’s better to stick with Python’s natural behavior and avoid relying on unsupported patterns.

[mrgolin]

I believe we can avoid this significant change, which will affect external Pyverbs users as well.

Agree this is a really big change that needs to be carefully reviewed, but as far as I see it should not break external Pyverbs users.

[Crazy-JY]

Hi, sorry to bother you~ I'm new with RDMA, but I met some problems when running rdma-core/tests/run_tests.py.
I cloned the latest version of the repo, but I got these kind of errors:

File "/home/rdma-core/tests/base.py", line 30, in <module>
    PATH_MTU = e.IBV_MTU_1024
AttributeError: module 'pyverbs.enums' has no attribute 'IBV_MTU_1024'

I think it should be e.ibv_mtu.IBV_MTU_1024 as you defined in libibverbs_enums.pxd.

I have no idea how to deal with it, cause it seems that e.XXXX is used many times.
Version of cython is 3.1.0. Should I change the version of rdma-core? thx~~

[YonatanNachum]

@Crazy-JY, Hey thats what this patch fixes. You can just cherry-pick it for now until it or a variation of it will be merged.

[Crazy-JY]

@Crazy-JY, Hey thats what this patch fixes. You can just cherry-pick it for now until it or a variation of it will be merged.

Oh Okay thx~ Looking forward to it~

[YonatanNachum]

Hey @EdwardSro how do you want to proceed with this change ? its fully backwards compatible and doesn't suppose to break older Cython versions so its suppose to be safe for any pyverbs users.

[EdwardSro]

I’m open to changing it if you strongly prefer that approach, but in my view, it’s better to stick with Python’s natural behavior and avoid relying on unsupported patterns.

On second thought, I tend to agree that it's better to stick with Python's natural behavior...
Also, shortening the enums, to some arbitrary names, would just be confusing.
Currently, I don't see a better choice than to fully adapt.

Agree this is a really big change that needs to be carefully reviewed, but as far as I see it should not break external Pyverbs users.

It does. Any external Pyverbs test/app needs to adapt to the new Python-like enums.
But, unfortunately, it's not the first time that it happens with Python and Cython upgrades.

Sorry for the delay, I'll do my best to fully review this patch in the upcoming days.