👷 Fix failing CI github action

[SimmonsRitchie]

Summary

Fixes a somewhat mysterious bug that has been causing CI builds to fail. This PR:

• Pins all the dependencies in our Pipfile
• Sets python 3.9 as a requirement in our Pipfile
• Rebuilds our Lock file.
• Makes two tweaks to import and invocation of ParserError in two files (explained below).

Why are we doing this?

We need a working CI process to catch bugs and linting issues before merging to production. I opened a PR with very minor changes the other day and immediately discovered our CI github action was failing. My PR only affected our readme. The error appeared unrelated to my changes and was occurring early, just as Pipenv attempted to install packages:

Successfully created virtual environment!
Virtualenv location: /home/runner/work/city-scrapers/city-scrapers/.venv
Installing dependencies from Pipfile.lock (172d69)...
[pipenv.exceptions.InstallError]: Collecting city-scrapers-core[azure]@ git+https://github.com/City-Bureau/city-scrapers-core.git@44ed57f1f4833f7c35276fff4bc5187e191da496 (from -r /tmp/pipenv-lnfh2pbt-requirements/pipenv-eloxw8jh-reqs.txt (line 1))
[pipenv.exceptions.InstallError]:   Cloning https://github.com/City-Bureau/city-scrapers-core.git (to revision 44ed57f1f4833f7c35276fff4bc5187e191da496) to /tmp/pip-install-rcissozi/city-scrapers-core_ee08d0b22bfb453b832816251185d8a1
[pipenv.exceptions.InstallError]:   Resolved https://github.com/City-Bureau/city-scrapers-core.git to commit 44ed57f1f4833f7c35276fff4bc5187e191da496
[pipenv.exceptions.InstallError]:   Installing build dependencies: started
[pipenv.exceptions.InstallError]:   Installing build dependencies: finished with status 'done'
[pipenv.exceptions.InstallError]:   Getting requirements to build wheel: started
[pipenv.exceptions.InstallError]:   Getting requirements to build wheel: finished with status 'done'
[pipenv.exceptions.InstallError]:   Preparing metadata (pyproject.toml): started
[pipenv.exceptions.InstallError]:   Preparing metadata (pyproject.toml): finished with status 'done'
[pipenv.exceptions.InstallError]: Collecting scrapy-sentry@ git+https://github.com/City-Bureau/scrapy-sentry.git@5da919f716d981e5c7926994f27676a285da39a3 (from -r /tmp/pipenv-lnfh2pbt-requirements/pipenv-eloxw8jh-reqs.txt (line 2))
[pipenv.exceptions.InstallError]:   Cloning https://github.com/City-Bureau/scrapy-sentry.git (to revision 5da919f716d981e5c7926994f27676a285da39a3) to /tmp/pip-install-rcissozi/scrapy-sentry_a286197c230d454db756ec714217ab69
[pipenv.exceptions.InstallError]:   Resolved https://github.com/City-Bureau/scrapy-sentry.git to commit 5da919f716d981e5c7926994f27676a285da39a3
[pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): started
[pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): finished with status 'error'
[pipenv.exceptions.InstallError]: Running command git clone --filter=blob:none --quiet https://github.com/City-Bureau/city-scrapers-core.git /tmp/pip-install-rcissozi/city-scrapers-core_ee08d0b22bfb453b832816251185d8a1
[pipenv.exceptions.InstallError]:   Running command git rev-parse -q --verify 'sha^44ed57f1f4833f7c35276fff4bc5187e191da496'
[pipenv.exceptions.InstallError]:   Running command git fetch -q https://github.com/City-Bureau/city-scrapers-core.git 44ed57f1f4833f7c35276fff4bc5187e191da496
[pipenv.exceptions.InstallError]:   Running command git clone --filter=blob:none --quiet https://github.com/City-Bureau/scrapy-sentry.git /tmp/pip-install-rcissozi/scrapy-sentry_a286197c230d454db756ec714217ab69
[pipenv.exceptions.InstallError]:   Running command git rev-parse -q --verify 'sha^5da919f716d981e5c7926994f27676a285da39a3'
[pipenv.exceptions.InstallError]:   Running command git fetch -q https://github.com/City-Bureau/scrapy-sentry.git 5da919f716d981e5c7926994f27676a285da39a3
[pipenv.exceptions.InstallError]:   Running command git checkout -q 5da919f716d981e5c7926994f27676a285da39a3
[pipenv.exceptions.InstallError]:   error: subprocess-exited-with-error
[pipenv.exceptions.InstallError]:   
[pipenv.exceptions.InstallError]:   × python setup.py egg_info did not run successfully.
[pipenv.exceptions.InstallError]:   │ exit code: 1
[pipenv.exceptions.InstallError]:   ╰─> [30 lines of output]
[pipenv.exceptions.InstallError]:       /home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/setuptools/__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
[pipenv.exceptions.InstallError]:       !!
[pipenv.exceptions.InstallError]:       
[pipenv.exceptions.InstallError]:               ********************************************************************************
[pipenv.exceptions.InstallError]:               Requirements should be satisfied by a PEP 517 installer.
[pipenv.exceptions.InstallError]:               If you are using pip, you can try `pip install --use-pep517`.
[pipenv.exceptions.InstallError]:               ********************************************************************************
[pipenv.exceptions.InstallError]:       
[pipenv.exceptions.InstallError]:       !!
[pipenv.exceptions.InstallError]:       Traceback (most recent call last):
[pipenv.exceptions.InstallError]:         File "<string>", line 2, in <module>
[pipenv.exceptions.InstallError]:         File "<pip-setuptools-caller>", line 34, in <module>
[pipenv.exceptions.InstallError]:         File "/tmp/pip-install-rcissozi/scrapy-sentry_a286197c230d454db756ec714217ab69/setup.py", line 3, in <module>
[pipenv.exceptions.InstallError]:           setup(
[pipenv.exceptions.InstallError]:         File "/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/setuptools/__init__.py", line 102, in setup
[pipenv.exceptions.InstallError]:         File "/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/setuptools/__init__.py", line 75, in _install_setup_requires
[pipenv.exceptions.InstallError]:         File "/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/setuptools/__init__.py", line 80, in _fetch_build_eggs
[pipenv.exceptions.InstallError]:         File "/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/setuptools/dist.py", line 636, in fetch_build_eggs
[pipenv.exceptions.InstallError]:         File "/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/setuptools/installer.py", line 38, in _fetch_build_eggs
[pipenv.exceptions.InstallError]:         File "/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 829, in resolve
[pipenv.exceptions.InstallError]:           dist = self._resolve_dist(
[pipenv.exceptions.InstallError]:         File "/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 865, in _resolve_dist
[pipenv.exceptions.InstallError]:           dist = best[req.key] = env.best_match(
[pipenv.exceptions.InstallError]:         File "/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1135, in best_match
[pipenv.exceptions.InstallError]:           return self.obtain(req, installer)
[pipenv.exceptions.InstallError]:         File "/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1147, in obtain
[pipenv.exceptions.InstallError]:           return installer(requirement)
[pipenv.exceptions.InstallError]:         File "/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/setuptools/installer.py", line 74, in _fetch_build_egg_no_warn
[pipenv.exceptions.InstallError]:         File "/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/setuptools/dist.py", line 677, in get_egg_cache_dir
[pipenv.exceptions.InstallError]:       ImportError: cannot import name 'windows_support' from 'setuptools' (/home/runner/work/city-scrapers/city-scrapers/.venv/lib/python3.9/site-packages/setuptools/__init__.py)
[pipenv.exceptions.InstallError]:       [end of output]
[pipenv.exceptions.InstallError]:   
[pipenv.exceptions.InstallError]:   note: This error originates from a subprocess, and is likely not a problem with pip.
[pipenv.exceptions.InstallError]: error: metadata-generation-failed
[pipenv.exceptions.InstallError]: 
[pipenv.exceptions.InstallError]: × Encountered error while generating package metadata.
[pipenv.exceptions.InstallError]: ╰─> See above for output.
[pipenv.exceptions.InstallError]: 
[pipenv.exceptions.InstallError]: note: This is an issue with the package mentioned above, not pip.
[pipenv.exceptions.InstallError]: hint: See above for details.
ERROR: Couldn't install package: {}
 Package installation failed...
Error: Process completed with exit code 1.

I tried a number of things to solve this issue and the best I could determine is that there was some kind of dependency issue in our Pipfile or pipfile.lock. I kept hitting errors whenever I tried to rebuild the lock file from the existing Pipfile.

Pinning dependencies and the Python version seems to have fixed things.

Are there any smells or added technical debt to note?

• Mystery bug: It's unclear to me whether it was one or multiple dependencies causing issues. It would be nice to know what the exact problem was because we don't have full assurance this PR fixes the underlying issue.
• Pinning: This PR pins our dependencies and there's some debate about whether it's good practice to pin dependencies. My general sense is that if this were a library then we should be wary, but as an application it's okay. I currently think it might be in our best interest to ensure this project is easy to reproduce by pinning our dependencies. Part of what made debugging this Github action issue difficult was trying to reproduce the same environment as the Github action. For this reason, I also tweaked our Pipfile to require Python 3.9. We're using 3.9 in each of the environments we're setting up in our our Github workflows so, to my mind, it makes sense to have no ambiguity about which version one should use when running this project locally. I hope it will avoid confusing errors (like this CI issue) down the line.