Can not install fresh cumulusci 3.80

[icukovic]

Describe the bug

Fails with following:

Fatal error from pip prevented installation. Full pip output in file:
    /Users/iuk/.local/pipx/logs/cmd_2023-10-09_16.30.45_pip_errors.log

pip seemed to fail to build package:
    pyyaml==6.0

Some possibly relevant errors from pip install:
    error: subprocess-exited-with-error
    AttributeError: cython_sources

Error log:

PIP STDOUT
----------
Collecting cumulusci
  Obtaining dependency information for cumulusci from https://files.pythonhosted.org/packages/ed/6b/bd59cbe88c56afbe2e9e819a2e6c076f6117589c1d3b928f88f3eb8dfbf7/cumulusci-3.80.0-py3-none-any.whl.metadata
  Using cached cumulusci-3.80.0-py3-none-any.whl.metadata (10 kB)
Collecting appdirs==1.4.4 (from cumulusci)
  Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting authlib==1.2.0 (from cumulusci)
  Using cached Authlib-1.2.0-py2.py3-none-any.whl (214 kB)
Collecting certifi==2022.12.7 (from cumulusci)
  Using cached certifi-2022.12.7-py3-none-any.whl (155 kB)
Collecting cffi==1.15.1 (from cumulusci)
  Using cached cffi-1.15.1.tar.gz (508 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting charset-normalizer==3.0.1 (from cumulusci)
  Using cached charset_normalizer-3.0.1-py3-none-any.whl (45 kB)
Collecting click==8.1.3 (from cumulusci)
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting cryptography==39.0.1 (from cumulusci)
  Using cached cryptography-39.0.1-cp36-abi3-macosx_10_12_universal2.whl (5.4 MB)
Collecting defusedxml==0.7.1 (from cumulusci)
  Using cached defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting docutils==0.16 (from cumulusci)
  Using cached docutils-0.16-py2.py3-none-any.whl (548 kB)
Collecting faker==17.0.0 (from cumulusci)
  Using cached Faker-17.0.0-py3-none-any.whl (1.7 MB)
Collecting fs==2.4.16 (from cumulusci)
  Using cached fs-2.4.16-py2.py3-none-any.whl (135 kB)
Collecting github3-py==3.2.0 (from cumulusci)
  Using cached github3.py-3.2.0-py2.py3-none-any.whl (152 kB)
Collecting greenlet==2.0.2 (from cumulusci)
  Using cached greenlet-2.0.2.tar.gz (164 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting gvgen==1.0 (from cumulusci)
  Using cached GvGen-1.0-py3-none-any.whl (8.0 kB)
Collecting idna==3.4 (from cumulusci)
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting importlib-metadata==6.0.0 (from cumulusci)
  Using cached importlib_metadata-6.0.0-py3-none-any.whl (21 kB)
Collecting jeepney==0.8.0 (from cumulusci)
  Using cached jeepney-0.8.0-py3-none-any.whl (48 kB)
Collecting jinja2==3.1.2 (from cumulusci)
  Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting keyring==23.0.1 (from cumulusci)
  Using cached keyring-23.0.1-py3-none-any.whl (33 kB)
Collecting lxml==4.9.2 (from cumulusci)
  Using cached lxml-4.9.2.tar.gz (3.7 MB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting markdown-it-py==2.1.0 (from cumulusci)
  Using cached markdown_it_py-2.1.0-py3-none-any.whl (84 kB)
Collecting markupsafe==2.1.2 (from cumulusci)
  Using cached MarkupSafe-2.1.2.tar.gz (19 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting mdurl==0.1.2 (from cumulusci)
  Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Collecting natsort==8.2.0 (from cumulusci)
  Using cached natsort-8.2.0-py3-none-any.whl (37 kB)
Collecting psutil==5.9.4 (from cumulusci)
  Using cached psutil-5.9.4-cp38-abi3-macosx_11_0_arm64.whl (244 kB)
Collecting pycparser==2.21 (from cumulusci)
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting pydantic==1.10.5 (from cumulusci)
  Using cached pydantic-1.10.5-py3-none-any.whl (155 kB)
Collecting pygments==2.14.0 (from cumulusci)
  Using cached Pygments-2.14.0-py3-none-any.whl (1.1 MB)
Collecting pyjwt[crypto]==2.6.0 (from cumulusci)
  Using cached PyJWT-2.6.0-py3-none-any.whl (20 kB)
Collecting python-baseconv==1.2.2 (from cumulusci)
  Using cached python-baseconv-1.2.2.tar.gz (4.9 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting python-dateutil==2.8.2 (from cumulusci)
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytz==2022.7.1 (from cumulusci)
  Using cached pytz-2022.7.1-py2.py3-none-any.whl (499 kB)
Collecting pyyaml==6.0 (from cumulusci)
  Using cached PyYAML-6.0.tar.gz (124 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'

PIP STDERR
----------
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [54 lines of output]
      running egg_info
      writing lib/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib/PyYAML.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/Users/iuk/.local/pipx/shared/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/iuk/.local/pipx/shared/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/iuk/.local/pipx/shared/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 355, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 288, in <module>
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 318, in run
          self.find_sources()
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 326, in find_sources
          mm.run()
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 548, in run
          self.add_defaults()
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 586, in add_defaults
          sdist.add_defaults(self)
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/command/sdist.py", line 113, in add_defaults
          super().add_defaults()
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 204, in get_source_files
        File "/private/var/folders/lz/wy2pbmfx2kx2102tgj6cggr00000gp/T/pip-build-env-p74jl9e0/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Reproduction steps

Fresh install accordingly to https://cumulusci.readthedocs.io/en/latest/get-started.html#verify-your-installation:

1. brew install pipx
2. pipx ensurepath
3. pipx install cumulusci

Your CumulusCI and Python versions

CumulusCi: N/A
Python2: 2.7.18
Python3: 3.12.0

Operating System

macOS Ventura 13.0.1

Windows environment

No response

CumulusCI installation method

pipx

Error Gist

No response

Additional information

No response

[uwreeser]

By any chance is there a workaround for this? I ended up here because a CI job in GitLab we have that builds a Docker image fails at the step where we're trying to install cci....instead it spits out essentially the same error noted here.

installing cumulusci...
Fatal error from pip prevented installation. Full pip output in file:
    /root/.local/pipx/logs/cmd_2023-10-11_17.54.37_1_pip_errors.log
pip seemed to fail to build package:
    pyyaml==6.0
Some possibly relevant errors from pip install:
    error: subprocess-exited-with-error
    AttributeError: cython_sources
Error installing cumulusci.

From what little I've been able to sort out I don't know if this is specifically a cumulusci issue or not? So I wasn't sure if its something I should wait to see if it gets resolved (soonish?) with CCI or if the solution lies elsewhere.....since the issue appears to be with pip failing to compile pyyaml because cython did something clever recently.

[mattsimonis]

I am running into the same thing and cannot get cci installed locally.

[jkinzer]

Seems downstream: yaml/pyyaml#736

[icukovic]

@uwreeser I am not sure either if this "bug" should belong here, but this prevents me from installing/using any version of cumulusci. I've tried installing previous versions without any success.

[mattsimonis]

A workaround for me to at least get the package installed locally to use was the following:

1. Clone repository
2. Update requirements/dev.txt and requirements/prod.txt to reference pyyaml==6.0.1 instead of 6.0
3. Run pipx install . in the base repository directory

[hkhandelwal-NCSDev]

@mattsimonis How to access requirements/dev.txt and requirements/prod.txt?

[davidjray]

@hkhandelwal-NCSDev after cloning this Repo to your computer you would update the requirements file to be pyyaml==6.0.1. The do the same with prod.txt.

Then you would install CumulusCI from the local files instead of from the remote package.

[icukovic]

Thanks @mattsimonis & @davidjray! It worked.

I am closing this ticket as there is a workaround for this.

[jkinzer]

@icukovic why not leave this open and have the maintainers upgrade the dependency? We shouldn't have to clone/modify a toolchain to make it work.

[icukovic]

@jkinzer Yea... I am causing a chaos here a little and I've closed it as I didn't know how the CumulusCI contributors/maintainers want to handle that one.

Reopening for now.

[jstvz]

Fixed by #3686

[jstvz]

You'll still hit that error until we cut the next release of CumulusCI.

As a workaround:

$ pipx install https://github.com/SFDO-Tooling/CumulusCI/archive/refs/heads/main.zip
  installed package cumulusci 3.80.0, installed using Python 3.12.0
  These apps are now globally available
    - cci
    - snowfakery
done! ✨ 🌟 ✨ 

Note: Python 3.12 won't be fully supported until #3691 is merged.