Please make the C extension build optional (e.g. via PURE_PYTHON)
#330
Comments
|
Michał Górny wrote at 2024-10-18 22:55 -0700:
## FEATURE REQUEST
Could you consider making the C extension build in `setup.py` optional, similarly to how the extensions are currently optional in tests? (i.e. if `PURE_PYTHON=1`, it wouldn't attempt to build them at all)
...
For the time being, we believe that the "lesser evil" would be to install install `zope.interface` without speedups, and keep GIL disabled for the remainder of the program. The way we see it, it would be done like:
```
$ PURE_PYTHON=1 pip install zope.interface
```
Why are you interested to suppress the installation of
the C extension?
Setting the envvar `PURE_PYTHON` to a value different from `"0"`
at run (in contrast to installation) time,
should prevent the C extension import.
Have you observed that this does not work?
If it really should not work, you can delete an installed C extension.
Binary distributions of `zope.interface` contain the C extension.
I fear that (new) special logic would be necessary to suppress its
installation based on the `PURE_PYTHON` envvar.
I am against the introduction of such special logic for your quite
special use case.
|
|
I'm packaging |
|
Michał Górny wrote at 2024-10-19 00:23 -0700:
I'm packaging `zope.interface` for Gentoo. Setting `PURE_PYTHON` at runtime does not resolve the more general problem that the package currently installs a harmful extension. Similarly, removing it would mean that users unnecessarily burn coal to build it. And the current design would mean we'd have to maintain an intrusive patch forever, vs. having to just control a single envvar.
There are distribution packagers who would like to omit the installation
of tests.
Apparently, they would prefer when the tests would not be included
in the distribution at all but can remove them without too much
problems during their own packaging.
Maybe, this would work as well for your `Gentoo` packaging regarding
the C optimazation.
|
|
Deleting |
|
Michał Górny wrote at 2024-10-19 01:38 -0700:
Deleting `.py` files that are merely copied is a minor cost compared to C extensions that are actually compiled, which involve nonzero energy cost.
Nevertheless, I would delegate the decision on whether to use
the C optimization to runtime.
I would follow logic
(in `zope.interface._compat._should_attempt_c_optimizations`)
used for `PyPy`. It could check whether it runs without GIL
and then not use the c extension.
Your `Gentoo` packaging would build the C extension
and it would be used when the GIL is used;
without GIL, it would not be used.
|
|
Hmm, I suppose that suggestion does make sense. Do you want me to make a patch? |
|
Michał Górny wrote at 2024-10-19 01:55 -0700:
Hmm, I suppose that suggestion does make sense. Do you want me to make a patch?
That would be nice!
To easily take over your patch, you would need to sign
the `Zope` contributor agreement.
|
Do not enable C speedups by default if running Python 3.13+ in freethreading mode (with GIL disabled). Since the extension does not support this mode explicitly at the moment, loading it would cause CPython to reenable GIL with a warning and penalize the whole environment. Discussed in issue zopefoundation#330.
Do not enable C speedups by default if running Python 3.13+ in freethreading mode (with GIL disabled). Since the extension does not support this mode explicitly at the moment, loading it would cause CPython to reenable GIL with a warning and penalize the whole environment. Discussed in issue zopefoundation#330.
We publish wheels with the extension already built, which covers 99.99% of all users of the package. Doing the extra work (if needed) when building for a Linux distribution isn't going to be measurable, compared to the efficiency of actually using the extension. WRT to the "declaration" that we support free-threading: while there is some work to be done to test / verfiy / declare that the package works under freethreading, we should be pretty close with the refactorings done to support heap-based types and multi-phase initialization. |
Do you have any honest, unbiased statistics to prove that number? Because 99.99% of the times when people say "99.99%", they're just making that up. |
|
Available download stats for
There are essentially three ways folks get
You'd be hard-pressed to convince me that there are a significant number of |
|
See #332 for work needed to support 3.13 FT. |
FEATURE REQUEST
Could you consider making the C extension build in
setup.pyoptional, similarly to how the extensions are currently optional in tests? (i.e. ifPURE_PYTHON=1, it wouldn't attempt to build them at all)Our primary use case right now is Python 3.13 freethreading. Since the speedups extension does not support freethreading, importing it effectively reenables GIL for the whole program:
For the time being, we believe that the "lesser evil" would be to install install
zope.interfacewithout speedups, and keep GIL disabled for the remainder of the program. The way we see it, it would be done like:The text was updated successfully, but these errors were encountered: