Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build the native extension during packaging #56

Open
kloczek opened this issue Nov 8, 2024 · 3 comments
Open

Build the native extension during packaging #56

kloczek opened this issue Nov 8, 2024 · 3 comments
Labels
Integrations - Python Technical Improvement

Comments

@kloczek
Copy link

kloczek commented Nov 8, 2024

build:

+ /usr/bin/python3 -sBm build -w --no-isolation
* Getting build dependencies for wheel...
* Building wheel...
Successfully built pytest_codspeed-3.0.0-py3-none-any.whl

and content of that .whl archive

 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
     234  Defl:N      136  42% 02-02-2020 00:00 98b28797  pytest_codspeed/__init__.py
   11790  Defl:N     3249  72% 02-02-2020 00:00 ca4e53b5  pytest_codspeed/plugin.py
       0  Defl:N        2   0% 02-02-2020 00:00 00000000  pytest_codspeed/py.typed
    1948  Defl:N      734  62% 02-02-2020 00:00 47524ea0  pytest_codspeed/utils.py
    1428  Defl:N      552  61% 02-02-2020 00:00 dd806002  pytest_codspeed/instruments/__init__.py
    7603  Defl:N     2119  72% 02-02-2020 00:00 298e6616  pytest_codspeed/instruments/walltime.py
    3113  Defl:N     1127  64% 02-02-2020 00:00 9ad99892  pytest_codspeed/instruments/valgrind/__init__.py
      36  Defl:N       38  -6% 02-02-2020 00:00 97dd7d97  pytest_codspeed/instruments/valgrind/_wrapper/.gitignore
     962  Defl:N      430  55% 02-02-2020 00:00 8e79d6b5  pytest_codspeed/instruments/valgrind/_wrapper/__init__.py
     392  Defl:N      188  52% 02-02-2020 00:00 dbd3797d  pytest_codspeed/instruments/valgrind/_wrapper/wrapper.c
     149  Defl:N       84  44% 02-02-2020 00:00 2f38678c  pytest_codspeed/instruments/valgrind/_wrapper/wrapper.h
     365  Defl:N      122  67% 02-02-2020 00:00 e1893709  pytest_codspeed/instruments/valgrind/_wrapper/wrapper.pyi
    5970  Defl:N     2453  59% 02-02-2020 00:00 8adba23d  pytest_codspeed-3.0.0.dist-info/METADATA
      87  Defl:N       86   1% 02-02-2020 00:00 74ffda86  pytest_codspeed-3.0.0.dist-info/WHEEL
      45  Defl:N       36  20% 02-02-2020 00:00 7ffc8e0e  pytest_codspeed-3.0.0.dist-info/entry_points.txt
    1092  Defl:N      650  41% 02-02-2020 00:00 533abf11  pytest_codspeed-3.0.0.dist-info/licenses/LICENSE
    1619  Defl:N      822  49% 02-02-2020 00:00 c7f2b1d6  pytest_codspeed-3.0.0.dist-info/RECORD
--------          -------  ---                            -------
   36833            12828  65%                            17 files

Additionally pytest execution causes building and linking DSO in install tree instead in source tree.

[tkloczko@pers-jacek python-pytest-codspeed-3.0.0-2.fc37.x86_64]$ ls -lR $PWD
/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64:
total 0
drwxr-xr-x 1 tkloczko tkloczko 6 Nov  8 22:04 usr

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr:
total 0
drwxr-xr-x 1 tkloczko tkloczko 20 Nov  8 22:04 lib

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib:
total 0
drwxr-xr-x 1 tkloczko tkloczko 26 Nov  8 22:04 python3.10

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10:
total 0
drwxr-xr-x 1 tkloczko tkloczko 92 Nov  8 22:04 site-packages

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages:
total 0
drwxr-xr-x 1 tkloczko tkloczko 116 Nov  8 22:04 pytest_codspeed
drwxr-xr-x 1 tkloczko tkloczko  86 Nov  8 22:04 pytest_codspeed-3.0.0.dist-info

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages/pytest_codspeed:
total 20
-rw-r--r-- 1 tkloczko tkloczko   234 Nov  8 22:04 __init__.py
drwxr-xr-x 1 tkloczko tkloczko   304 Nov  8 22:04 __pycache__
drwxr-xr-x 1 tkloczko tkloczko    82 Nov  8 22:04 instruments
-rw-r--r-- 1 tkloczko tkloczko 11790 Nov  8 22:04 plugin.py
-rw-r--r-- 1 tkloczko tkloczko     0 Nov  8 22:04 py.typed
-rw-r--r-- 1 tkloczko tkloczko  1948 Nov  8 22:04 utils.py

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages/pytest_codspeed/__pycache__:
total 40
-rw-r--r-- 1 tkloczko tkloczko   289 Nov  8 22:04 __init__.cpython-310.opt-1.pyc
-rw-r--r-- 1 tkloczko tkloczko   289 Nov  8 22:04 __init__.cpython-310.pyc
-rw-r--r-- 1 tkloczko tkloczko 10142 Nov  8 22:04 plugin.cpython-310.opt-1.pyc
-rw-r--r-- 1 tkloczko tkloczko 10156 Nov  8 22:04 plugin.cpython-310.pyc
-rw-r--r-- 1 tkloczko tkloczko  2192 Nov  8 22:04 utils.cpython-310.opt-1.pyc
-rw-r--r-- 1 tkloczko tkloczko  2192 Nov  8 22:04 utils.cpython-310.pyc

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages/pytest_codspeed/instruments:
total 12
-rw-r--r-- 1 tkloczko tkloczko 1428 Nov  8 22:04 __init__.py
drwxr-xr-x 1 tkloczko tkloczko  216 Nov  8 22:04 __pycache__
drwxr-xr-x 1 tkloczko tkloczko   60 Nov  8 22:04 valgrind
-rw-r--r-- 1 tkloczko tkloczko 7603 Nov  8 22:04 walltime.py

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages/pytest_codspeed/instruments/__pycache__:
total 24
-rw-r--r-- 1 tkloczko tkloczko 2437 Nov  8 22:04 __init__.cpython-310.opt-1.pyc
-rw-r--r-- 1 tkloczko tkloczko 2437 Nov  8 22:04 __init__.cpython-310.pyc
-rw-r--r-- 1 tkloczko tkloczko 7111 Nov  8 22:04 walltime.cpython-310.opt-1.pyc
-rw-r--r-- 1 tkloczko tkloczko 7111 Nov  8 22:04 walltime.cpython-310.pyc

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages/pytest_codspeed/instruments/valgrind:
total 4
-rw-r--r-- 1 tkloczko tkloczko 3113 Nov  8 22:04 __init__.py
drwxr-xr-x 1 tkloczko tkloczko  108 Nov  8 22:04 __pycache__
drwxr-xr-x 1 tkloczko tkloczko  306 Nov  8 22:04 _wrapper

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages/pytest_codspeed/instruments/valgrind/__pycache__:
total 8
-rw-r--r-- 1 tkloczko tkloczko 3342 Nov  8 22:04 __init__.cpython-310.opt-1.pyc
-rw-r--r-- 1 tkloczko tkloczko 3342 Nov  8 22:04 __init__.cpython-310.pyc

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages/pytest_codspeed/instruments/valgrind/_wrapper:
total 176
-rw-r--r-- 1 tkloczko tkloczko   962 Nov  8 22:04 __init__.py
drwxr-xr-x 1 tkloczko tkloczko   108 Nov  8 22:04 __pycache__
-rw-r--r-- 1 tkloczko tkloczko     0 Nov  8 22:04 build.lock
-rw-r--r-- 1 tkloczko tkloczko 26786 Nov  8 22:04 dist_callgrind_wrapper.c
-rw-r--r-- 1 tkloczko tkloczko 72104 Nov  8 22:04 dist_callgrind_wrapper.o
-rwxr-xr-x 1 tkloczko tkloczko 34672 Nov  8 22:04 dist_callgrind_wrapper.so
-rw-r--r-- 1 tkloczko tkloczko   392 Nov  8 22:04 wrapper.c
-rw-r--r-- 1 tkloczko tkloczko   149 Nov  8 22:04 wrapper.h
-rw-r--r-- 1 tkloczko tkloczko 22272 Nov  8 22:04 wrapper.o
-rw-r--r-- 1 tkloczko tkloczko   365 Nov  8 22:04 wrapper.pyi

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages/pytest_codspeed/instruments/valgrind/_wrapper/__pycache__:
total 8
-rw-r--r-- 1 tkloczko tkloczko 1252 Nov  8 22:04 __init__.cpython-310.opt-1.pyc
-rw-r--r-- 1 tkloczko tkloczko 1252 Nov  8 22:04 __init__.cpython-310.pyc

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages/pytest_codspeed-3.0.0.dist-info:
total 20
-rw-r--r-- 1 tkloczko tkloczko 5970 Nov  8 22:04 METADATA
-rw-r--r-- 1 tkloczko tkloczko 1619 Nov  8 22:04 RECORD
-rw-r--r-- 1 tkloczko tkloczko   87 Nov  8 22:04 WHEEL
-rw-r--r-- 1 tkloczko tkloczko   45 Nov  8 22:04 entry_points.txt
drwxr-xr-x 1 tkloczko tkloczko   14 Nov  8 22:04 licenses

/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-codspeed-3.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages/pytest_codspeed-3.0.0.dist-info/licenses:
total 4
-rw-r--r-- 1 tkloczko tkloczko 1092 Nov  8 22:04 LICENSE

So it pollutes python sitelib tree by installing DSOs.
@kloczek
Copy link
Author

kloczek commented Nov 8, 2024

I'm not 100% sure but looks like valgrind DSO instrumentation is loaded from pytest_codspeed/instruments__init__.py so looks like building that DSO should be done on pep517 build procedure and that DSO should be part of .whl

@art049
Copy link
Member

art049 commented Nov 19, 2024 •
edited
Loading

Hi, we're building the native extension at runtime: https://github.com/CodSpeedHQ/pytest-codspeed/blob/master/src/pytest_codspeed/instruments/valgrind/_wrapper/__init__.py

But yes, we could definitely build it while packaging if that's what you meant. Happy if you want to make a PR. Otherwise, we'll tackle this when we have time.

@art049 art049 changed the title 3.0.0: pep517 build packs into .whl .c, .h, .pyi and .gitignore files Build the native extension during packaging Nov 19, 2024
@kloczek
Copy link
Author

kloczek commented Nov 19, 2024

Happy if you want to make a PR otherwise, we'll tackle this when we have time.

Sorry I'm not familiar with building DSO modules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Integrations - Python Technical Improvement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@art049 @kloczek