Project fails to build under pep517

[jaraco]

If I attempt to build one of the plugins under pep517, it fails with this traceback:

root@bf29a9e52d27:/# pip install --use-pep517 'pytest-virtualenv@ git+https://github.com/man-group/pytest-plugins#subdirectory=pytest-virtualenv'
Collecting pytest-virtualenv@ git+https://github.com/man-group/pytest-plugins#subdirectory=pytest-virtualenv
  Cloning https://github.com/man-group/pytest-plugins to /tmp/pip-install-n5u0efiw/pytest-virtualenv_59331a8cc84045d8b8b1bacc5cdb9307
  Running command git clone -q https://github.com/man-group/pytest-plugins /tmp/pip-install-n5u0efiw/pytest-virtualenv_59331a8cc84045d8b8b1bacc5cdb9307
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3.9 /usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpj2_gha2n
       cwd: /tmp/pip-install-n5u0efiw/pytest-virtualenv_59331a8cc84045d8b8b1bacc5cdb9307/pytest-virtualenv
  Complete output (18 lines):
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
      main()
    File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/_in_process.py", line 114, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-g15tkrb_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
      return self._get_build_requires(
    File "/tmp/pip-build-env-g15tkrb_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 143, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-g15tkrb_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 267, in run_setup
      super(_BuildMetaLegacyBackend,
    File "/tmp/pip-build-env-g15tkrb_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 158, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 6, in <module>
      from common_setup import common_setup
  ModuleNotFoundError: No module named 'common_setup'
  ----------------------------------------
WARNING: Discarding git+https://github.com/man-group/pytest-plugins#subdirectory=pytest-virtualenv. Command errored out with exit status 1: /usr/bin/python3.9 /usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpj2_gha2n Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement pytest-virtualenv (unavailable)
ERROR: No matching distribution found for pytest-virtualenv (unavailable)
WARNING: You are using pip version 20.3.4; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3.9 -m pip install --upgrade pip' command.

Pep517 uses isolated builds, copying the source tree for installation. Because these plugins require code found outside the source tree, it fails.

[pradyunsg]

Does this fail with pip 21.3?

[jaraco]

Yes, with 21.3.1 produces the same error.

[jaraco]

$ pip-run --use-pep517 'pytest-virtualenv@ git+https://github.com/man-group/pytest-plugins#subdirectory=pytest-virtualenv'
Collecting pytest-virtualenv@ git+https://github.com/man-group/pytest-plugins#subdirectory=pytest-virtualenv
  Cloning https://github.com/man-group/pytest-plugins to /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-0ojrwvyf/pytest-virtualenv_95ee5060e07c408db4f8f4ac625e4296
  Running command git clone --filter=blob:none -q https://github.com/man-group/pytest-plugins /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-0ojrwvyf/pytest-virtualenv_95ee5060e07c408db4f8f4ac625e4296
  Resolved https://github.com/man-group/pytest-plugins to commit 2929621566d6510849878d913da6509635e666e1
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10 /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpu85pcbio
       cwd: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-0ojrwvyf/pytest-virtualenv_95ee5060e07c408db4f8f4ac625e4296/pytest-virtualenv
  Complete output (18 lines):
  Traceback (most recent call last):
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
      main()
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-env-vcfdxfks/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
      return self._get_build_requires(
    File "/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-env-vcfdxfks/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 143, in _get_build_requires
      self.run_setup()
    File "/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-env-vcfdxfks/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 267, in run_setup
      super(_BuildMetaLegacyBackend,
    File "/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-env-vcfdxfks/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 158, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 6, in <module>
      from common_setup import common_setup
  ModuleNotFoundError: No module named 'common_setup'
  ----------------------------------------
WARNING: Discarding git+https://github.com/man-group/pytest-plugins#subdirectory=pytest-virtualenv. Command errored out with exit status 1: /Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10 /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpu85pcbio Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement pytest-virtualenv (unavailable) (from versions: 1.0.0, 1.0.1, 1.1.0, 1.2.2, 1.2.4, 1.2.5, 1.2.7, 1.2.11, 1.3.0, 1.4.0, 1.6.0, 1.7.0)
ERROR: No matching distribution found for pytest-virtualenv (unavailable)

[eeaston]

Hi @jaraco , thanks for raising this.

pip-run is a new one on me, I'll have look at adjusting the setup files such that the common files are checked in rather than just copied as part of the top-level Makefile.

[jaraco]

pip-run is just like pip install, except that it's transient (if it succeeds, it doesn't change anything about the environment, so you don't have to worry about creating an environment or polluting your current one), but feel free to use pip install to test/replicate.

[SteveHespelt]

Hoping this comment might be a little helpful - I started working rather recently with pytest-virtualenv, pytest-profiling in a fork. Ran into this issue immediately (without the --use-ppe517 flag being used) because I hadn't read the Makefile, hadn't used the make copyfiles target. And of course, the pip517 support won't 'know' to use make copyfiles. I'm pretty new to the exciting world of Python but I'd be tempted to leverage git's support for links, for this use case of 1 golden copy of the common_setup.py, VERSION, etc. files that are being 'shared' across the various pytest-plugins/* directories. Of course, I'm assuming (nothing mentioned in the top-level README.md) these files, once copied to the subdirectories, are not meant to diverge in their content. If I'm wrong, I apologize for my assumption. I'm also assuming links for those files ought to be viable (even on platforms where git will create regular files, that's ok too - after all, that's what make copyfiles does but only when manually invoked)

I'm a big proponent of keeping a source tree compatible with tools. As Jason's initial post shows, that's not possible. My suggestion: keep the Makefile as it is (since after the clone/fetch/pull, updating 1 of the golden copy on a non-link platform, we'd want to 'push' those changes to the subdirectories for inclusion in subsequent builds), but go with links so the subdirectories can be immediately built, installed, as in Jason's initial pip install above. If links aren't possible, I'd value learning as I'd rather not assume incorrectly ;-)

as in my fork, branched off of 1.7.1, once i use make copyfiles, pip install --use-pep517 . works for me. But that's off of commit 75e04eb.