Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Detect Python version from poetry-managed pyproject.toml #8

Open
wants to merge 20 commits into
base: master
Choose a base branch
from

Conversation

cjolowicz
Copy link
Owner

@cjolowicz cjolowicz commented Aug 23, 2019

This PR is a follow-up to #7.

If no runtime.txt is present, generate it using the pyproject.toml file in a Poetry-managed project. The Python version is specified in the section [tool.poetry.dependencies] under the python key.

We only handle a few simple cases here. If the Python version constraints are any more complicated, the author needs to provide an explicit runtime.txt.

Note that unlike pipenv, Poetry does not include an exact Python version in its lock file. The most we can do is pick a supported Python runtime that satisfies the constraints specified in pyproject.toml.

@cjolowicz cjolowicz force-pushed the ci-poetry-python-version branch from dff842a to 3bfd5a5 Compare August 23, 2019 19:41
@cjolowicz cjolowicz changed the title Detect Python version from poetry-managed pyproject.toml WIP: Detect Python version from poetry-managed pyproject.toml Aug 23, 2019
@cjolowicz cjolowicz force-pushed the ci-poetry-python-version branch 6 times, most recently from e1aa6db to d091254 Compare August 25, 2019 06:33
@cjolowicz cjolowicz force-pushed the ci-poetry-python-version branch 8 times, most recently from 3aa3851 to 1d81c44 Compare August 29, 2019 08:05
@cjolowicz cjolowicz force-pushed the ci-poetry-python-version branch from 1d81c44 to 957b570 Compare August 29, 2019 09:12
@cjolowicz cjolowicz force-pushed the ci-poetry-python-version branch 2 times, most recently from 8ef2fba to e90b9db Compare September 25, 2019 18:21
@cjolowicz cjolowicz force-pushed the ci-poetry-python-version branch from e90b9db to 01e023e Compare October 3, 2019 15:03
Change the HEROKU_API_KEY and HEROKU_API_USER environment variables used by
Travis CI. The encrypted values were generated using the following commands:

  heroku authorizations:create --description "For Travis"
  travis login --com
  travis encrypt HEROKU_API_KEY=<token> --add --com
  travis encrypt HEROKU_API_USER=<[email protected]> --add --com
@cjolowicz cjolowicz force-pushed the ci-poetry-python-version branch from 01e023e to 03608b9 Compare November 24, 2019 14:03
Run the tests in a separate CI job per test suite and stack. Testing on
all stacks in a single job exceeds Travis time limits.
Make it possible to specify individual tests to run, to facilitate
debugging. Running the entire test suite every time can take quite long.
Tests can be specified by passing the TESTS variable to `make test`.

If TESTS is non-empty, the test script defines the `suite` hook to build
the test suite using `suite_addTest` with each specified test function.
If TESTS is unset or empty, shUnit2's default behaviour is to run all
functions beginning with the word `test`.

See https://github.com/kward/shunit2#-suites
Do not attempt to invoke test functions specified via the TESTS
environment variable if they are not defined. The Makefile invokes
multiple test scripts and each test function is only defined in one of
them.
Define a noop test function when TESTS is passed and no function in the
test script matches the contents of TESTS. The noop function merely
prints a line notifying the user that the tests were skipped.

This happens because tests were split up into multiple test scripts, but
the Makefile passes TESTS to all of them. When the `suite` hook does not
invoke `suite_addTest`, shunit2 falls back to running all defined tests.
@cjolowicz cjolowicz force-pushed the ci-poetry-python-version branch from b836258 to 971ceca Compare November 24, 2019 14:25
Bump PIP_UPDATE from 9.0.2 to 19.2.3. This variable is used in bin/steps/python
to determine which pip version to install or upgrade to.
Python 3.4 support was dropped in pip >= 19.2. For projects still on
this Python version, use pip 19.1.1 instead of pip 19.2.1.
The pip-diff and pip-grep tools from the vendorized `pip-pop` package
import internal modules from pip. In pip >= 10, internal modules were
moved under `pip._internal`, breaking the imports. Use `try...except
ImportError` to handle both import paths.

Also, the interface of the `PackageFinder` class from one of these
modules changed. Provide a wrapper function to allow creating objects of
this type using the old interface.
This addresses an issue raised by @CaseyFeist during code review:

  Updating pip for pipenv users or requiring them to update without a
  heads up won't be a good experience (our version is old enough that
  they'll need to uninstall and reinstall pipenv locally to successfully
  update). If you can refactor this to stay pinned to current version for
  pipenv users only, I should be able to accept this (and the related
  project updates).

  heroku#833 (comment)
Add two test cases for projects using poetry as the build backend:

- testPoetry: project without dependencies

- testPoetryLock: project with dependencies, using a poetry.lock file
Add two test cases for projects using flit as the build backend:

- testFlit: project without dependencies

- testFlitRequires: project with dependencies
Check that pinned requirements are honored when poetry.lock is present.
Request an outdated version of marshmallow in poetry.lock (3.0.0). Check
that the outdated version is installed, rather than a newer version
also satisfying the version constraint in pyproject.toml (^3.0.0).
Install pyproject.toml-based projects using `pip install .`. Do not use a
requirements.txt for this. The requirements.txt file is needed to handle pinned
versions extracted from the poetry.lock file.

Note that we cannot simply append `.` to the exported requirements.txt file.
Currently pip requires that either all requirements have a hash or none.
Including `.` would thus force us to omit hashes for all requirements.
If a poetry.lock file is present, use the Poetry CLI to export the
pinned requirements to a requirements.txt file. If the project already
contains a requirements.txt, use that and ignore poetry.lock.

Poetry is not used to install the project because it does not clean up
stale requirements. This means that requirements need to be exported
anyway, for the `pip-uninstall` step.

Since we only use Poetry to export a requirements.txt file, ignore the
Poetry version specified in pyproject.toml. Install a pre-release of
1.0.0 because the export command is not available before 1.0.0a0.

Note that supporting pyproject.toml-based installations is not enough to
handle pinned requirements: When pip installs a pyproject.toml-style
project using the process described in PEP 517, it only uses Poetry to
build a wheel. The wheel contains the version constraints from
pyproject.toml, not the pinned versions from poetry.lock.
If no runtime.txt is present, generate it using the pyproject.toml file
generated by Poetry. The Python version is specified in the section
`[tool.poetry.dependencies]` under the `python` key.

We only handle a few simple cases here. If the Python version
constraints are any more complicated, the user needs to provide an
explicit runtime.txt.
@cjolowicz cjolowicz force-pushed the ci-poetry-python-version branch from 971ceca to 0cffcab Compare November 24, 2019 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant