CI tests failing with pypmc build error

[titodalcanton]

Tests have been failing in the last few days with an error that goes like this:

py-unittest installdeps: :preinstall:-rrequirements.txt, -rcompanion.txt, mkl;'arm' not in platform_machine, pytest, git+https://github.com/titodalcanton/BBHx.git@py39-and-cleanup; sys_platform == 'linux', git+https://github.com/gwastro/BBHX-waveform-model.git; sys_platform == 'linux'
ERROR: invocation failed (exit code 1), logfile: /home/runner/work/pycbc/pycbc/.tox/py-unittest/log/py-unittest-3.log
================================== log start ===================================
[many lines dropped by me]
INFO: pip is looking at multiple versions of pypmc to determine which version is compatible with other requirements. This could take a while.
Collecting pypmc (from snowline->-r companion.txt (line 21))
  Downloading pypmc-1.2.1.tar.gz (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 115.1 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
  Downloading pypmc-1.2.tar.gz (979 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 979.5/979.5 kB 88.1 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
  Downloading pypmc-1.1.4.tar.gz (1.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 53.1 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [27 lines of output]
      Traceback (most recent call last):
        File "/home/runner/work/pycbc/pycbc/.tox/py-unittest/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/runner/work/pycbc/pycbc/.tox/py-unittest/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/runner/work/pycbc/pycbc/.tox/py-unittest/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-xnsv304b/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 373, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/tmp/pip-build-env-xnsv304b/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 503, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-xnsv304b/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 318, in run_setup
          exec(code, locals())
        File "<string>", line 91, in <module>
        File "<string>", line 85, in setup_package
        File "<string>", line 37, in get_extensions
        File "/tmp/pip-build-env-xnsv304b/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1010, in cythonize
          module_list, module_metadata = create_extension_list(
                                         ^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-xnsv304b/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 845, in create_extension_list
          for file in nonempty(sorted(extended_iglob(filepattern)), "'%s' doesn't match any files" % filepattern):
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-xnsv304b/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 117, in nonempty
          raise ValueError(error_msg)
      ValueError: 'pypmc/*/*.pyx' doesn't match any files

This problem seems to happen only on tests running under Python ≥3.10; 3.9 works fine. There have been no recent releases of pypmc or snowline that could explain this. I can reproduce the error locally by running tox from inside PyCBC's working copy, but it is not obviously reproducible by just installing snowline on its own, so it must be something quite subtle. Also, @lpsinger noted in #4829 that

It's failing to install pypmc from source, but it's trying to build an old version when there are newer ones with wheels available.

Indeed, in my local tests with Python 3.12 I always get pypmc 1.2.2 (even though 1.2.3 is available) while the CI test with Python 3.9 gets 1.2.3! Looking at the diff between pypmc 1.2.2 and 1.2.3, I cannot understand what could cause 1.2.2 to be preferred to 1.2.3.

[spxiwh]

I'll give this a few days to see if someone is actively using snowline and wants to fix this. If not, I'll (or @GarethCabournDavies should if he gets to it first) remove it from companion.txt (with a note to reenable if this can be fixed).

[titodalcanton]

I see that pypmc 1.2.3 does not have a source file on PyPI, and none of the wheels there support Python 3.12. So at least we know why we do not get 1.2.3 under Python 3.12…

pypmc/pypmc#104

[titodalcanton]

If I force PyCBC to depend on pypmc 1.2.2 in companion.txt, then the CI tests seem to pass! See #4862.