Skip to content

Latest commit

 

History

History
145 lines (117 loc) · 5.25 KB

py2-migration.md

File metadata and controls

145 lines (117 loc) · 5.25 KB

Python 2

Dependencies: Last Version still supporting Python2

Py2 support was dropped by Pypi a while ago. Since then, when installing packages in a Py2 environment, pip does not even care to pick a version of the library that works on Py2. It just installs the latest available for Py3.

So, while we still have to support those legacy enviroments, we'll have to pin the versions of our dependencies and even the indirect dependencies.

The following list are the libraries we found stopped supporting Py2, and the required version pinning to get them working.

'arrow<1', # Py2, indirect empowering
'attrs<22', # Py2, indirect of pytest
'babel<=2.9.0', # Py2, indirect of flask-babel
'cachelib<0.2', # Py2
'click<8', # Py2, indirect pytest, flask
'configparser<5', # Py2, indirect of importlib-metadata, of pytest
'connexion<2.5.0', # Py2
'contextlib2<21', # Py2, indirect by many libs
'coverage<6', # Py2, indirect of pytest-cov
'coveralls<2.0', # Py2, direct dependency of the CI
'cryptography<3.4', # Py2, indirect of coveralls
'decorator<5', # Py2
'flask<2', # Py2
'flask-babel<2', # Py2
'importlib-metadata<3', # Py2, indirect of pytest
'jsonschema<4', # Py2
'MarkupSafe<2', # Py2, indirect jinja2 <- flask
'marshmallow<3', # Py2, indirect empowering
'mock<4', # Py2, indirect from pytest
'numpy<1.17', # Py2
'openapi-schema-validator<0.2', # Py2, indirect of openapi-spec-validator
'openapi-spec-validator<0.4', # Py2, indirect of connexion
'packaging<21', # Py2, indirect importlib-metadata, zipp
'platformdirs<2.1', # Py2, indirect of zipp
'pymongo<4', # Py2, indirect flask-pymongo
'pyopenssl<22', # Py2,  indirect of cryptography
'pyparsing<3', # Py2, indirect of packaging, of pytest
'pyrsistent<0.17', # Py2, indirect of jsonschema
'pytest<5', # Py2 support dropped
'pytest-cov<3', # Py2
'python-dotenv<0.19', # Py2
'pyyaml<6', # Py2, indirect of many
'redis<4', # Py2, indirect rq
'requests<2.28', # Py2, indirect of zipp, of pytest
'rq<1.4.0', # Py2, indirect amoniak
'sentry-sdk<1.5', # Py2
'tqdm<4.63.0', # Py2, depends on import-resources, not supported by py2
'urllib3<2', # Py2, indirect by request, django, flask...
'yamlns>=0.3', # Earlier not Py2 compatible
'zipp<2', # Py2, indirect of pytest
'rq-scheduler<0.12', # Py2, indirect erp-gisce/addons/base
'parameterized<0.9', # Py2

Annotate your pinned dependencies

Annotating the reason for a pinned dependency lets your coworkers feel empowered to remove the pinning when the reason does not apply anymore. In this case, we suggest using "Py2" to annotate those pinnings, so it could be dropped when Py2 is no longer supported.

For the same reason, it is useful to annotate whether it is a direct or indirect dependency. For a direct dependency, you just drop the pinning, but indirect dependencies should be fully dropped once in Py3.

How to include that?

In setup.py

import sys
py2 = sys.version_info<(3,)

Then try to differentiate direct dependencies and indirect.

  • For direct, keep the alternative for Py3.
  • For indirect, the alternative should be an empty string and try to keep track of the direct dependency that leads to it.
    'pytest<5' if py2 else 'pytest' # Py2
    'zipp<2' if py2 else '' # Py2, indirect of pytest

As requirements

pytest<5; python_version < '3.0'
pytest; python_version >= '3.0'
zipp<2; python_version < '3.0'

Caution, if your setup.py reads requirements.txt in order to fill install_requires, this would break setup.py, since this syntax of conditional dependencies is not supported by setuptools v44, the last one available for Py2. In this case you coul do a third approach.

As requirements also used in setup.py

In the case you reuse the dependencies.txt from setup.py to avoid duplication:

  • Place your regular dependencies in requirements.txt
pytest
  • Place the pinned version in requirements-py2.txt
pytest<5 # Py2
zipp<2 # Py2, indirect pytest
  • Modify the setup.py to load one or another depending on the current python version.
import sys
py2 = sys.version_info<(3,)                                                       
requirements_file = 'requirements-py2.txt' if py2 else 'requirements.txt'
requirements = file(requirements_file).open().readlines()
with open(requirements_file, 'r') as req:
    install_requires = [x.strip() for x in req.readlines()]

How to know the pinned version for a new library dropping Py2?

Local development environments usually do not detect such incompatibilities because if a library is installed it won't be updated by default. In order to trigger and clean those errors you should use a clean environment.

  • Run the line
$ deactivate ; rmvirtualenv py2; mkvirtualenv --python $(which python2) py2; pip install pipdeptree; ./setup.py develop
  • Look in the log for the last library starting its installation, is usually the problematic one.
  • Run pipdeptree to see which packages trigger the installation of the problematic one, in order to annotate indirect dependency path
  • Search the package in https://pypi.org
  • Go to the repository and open the CHANGES file
  • Tip: Look for "Python 2" or "Python" in order to see which is the last version suporting Python 2.
  • Start over again until it fully works