Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for pyproject.toml #21

Closed
jugmac00 opened this issue Oct 18, 2020 · 17 comments · Fixed by #34
Closed

Support for pyproject.toml #21

jugmac00 opened this issue Oct 18, 2020 · 17 comments · Fixed by #34
Labels
enhancement New feature or request

Comments

@jugmac00
Copy link
Contributor

> git clone [email protected]:jedie/python-creole.git
> cd python-creole
> check-python-versions
no setup.py -- not a Python package?

P.S.: This is just the information, that currently check-python-versions does not support projects which only use a pyproject.toml and no setup.py, which maybe a future trend.

At the current time, personally, I do not need this support and I still think using the common setup.py approach is superior.

@mgedmin mgedmin added the enhancement New feature or request label Oct 18, 2020
@mgedmin
Copy link
Owner

mgedmin commented Oct 18, 2020

How does one declare supported Python versions in a pyproject.toml?

(Trick question! pyproject.toml allows one to use any build backend, and that build backend can declare classifiers/python_requires any way it wants! I'll definitely want to support the more popular ones at least. E.g. the declarative version of setuptools where classifiers and requires_python are spelled out in a setup.cfg (#8). Ideally people will send me PRs.)

On a side note, I've recently wanted to use check-python-versions on a Vim plugin that had a tox.ini and a .travis.yml but no setup.py. I definitely want to relax the "must have a setup.py" requirement.

@billsioros
Copy link

Any news on this ? This affects cookiecutter templates as well!

@mgedmin
Copy link
Owner

mgedmin commented Sep 14, 2021

No news yet.

@gpongelli
Copy link
Contributor

How does one declare supported Python versions in a pyproject.toml?

for poetry users, the pyproject.toml file contains classifier array into tool.poetry section :

[tool.poetry]
    name = "..."
    classifiers=[
        'Development Status :: 2 - Pre-Alpha',
        'Intended Audience :: Developers',
        'Natural Language :: English',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.9',
        'Programming Language :: Python :: 3.10',
    ]

poetry docs

ps
for poetry, this package could be implemented also as poetry plugin (to call as poetry run check-python-version ... )

Regards.

@mgedmin
Copy link
Owner

mgedmin commented Feb 6, 2023

So turns out there's a spec (PEP 621) that should be suitable for all project backends. The canonical spec location is https://packaging.python.org/en/latest/specifications/declaring-project-metadata/#declaring-project-metadata.

project.classifiers and project.requires-python are the fields relevant to check-python-versions.

The PEP also links to flit/setuptools/poetry documentation. E.g. flit supports old-style metadata (tool.flit.metadata.classifiers) in addition to PEP 621 metadata.

@mgedmin
Copy link
Owner

mgedmin commented Feb 7, 2023

Having noticed some issues with the merged PR I'm now fixing them:

  • tool.poetry.dependencies.python uses different syntax and thus the current support is incorrect (I plan to remove it instead of fixing because I haven't the energy to implement full support)
  • both update functions strip away all the newlines from the files
  • old-style flit metadata is not supported, but adding support for it is easy (just use a different table: tool.flit.metadata instead of project)

@mgedmin mgedmin reopened this Feb 7, 2023
@gpongelli
Copy link
Contributor

  • tool.poetry.dependencies.python uses different syntax and thus the current support is incorrect (I plan to remove it instead of fixing because I haven't the energy to implement full support)

could you create the tests on a branch? I can try to fix it when I have some time.

  • both update functions strip away all the newlines from the files

could you create the tests on a branch? I can try to fix it when I have some time.

  • old-style flit metadata is not supported, but adding support for it is easy (just use a different table: tool.flit.metadata instead of project)

I didn't add it because new-style was introduced since flit 3.2 (two years ago), and actually there is flit 3.8 . if anyone still uses the old-style, an issue will be opened.

@mgedmin
Copy link
Owner

mgedmin commented Feb 7, 2023

tool.poetry.dependencies.python uses different syntax and thus the current support is incorrect

I am a complete idiot ant the previous statement is incorrect.

@mgedmin
Copy link
Owner

mgedmin commented Feb 7, 2023

tool.poetry.dependencies.python uses different syntax and thus the current support is incorrect

I am a complete idiot ant the previous statement is incorrect.

The first half of this statement is true, the second is false.

While both project.requires-python and tool.poetry.dependencies.python contain something called a "version specifier", the first uses PEP-440, while the second uses something else.

@mgedmin
Copy link
Owner

mgedmin commented Feb 7, 2023

could you create the tests on a branch? I can try to fix it when I have some time.

I've created a test marked with @pytest.mark.xfail in git master, specifically

@pytest.mark.xfail(
reason="Poetry-style Python requirements need implementation")
def test_get_python_requires_poetry(tmp_path, fix_max_python_3_version):
fix_max_python_3_version(9)
pyproject_toml = tmp_path / "pyproject.toml"
pyproject_toml.write_text(textwrap.dedent("""\
[tool.poetry.dependencies]
python = "^3.8"
"""))
assert get_python_requires(pyproject_toml) == v(['3.8', '3.9'])

I'd appreciate some assistance for making it pass. I fear that we either need to implement a parser for the Poetry dependency specification format, or find a library we can import and reuse.

The new code should live in parsers/poetry_version_spec.py (unless there's a better known name for this format? I think NPM uses it too). It should roughly mirror the API that parsers/requires_python.py exposes.

If I have that, I can finish hooking it up inside sources/pyproject.py myself.

@mgedmin
Copy link
Owner

mgedmin commented Feb 7, 2023

Hmm, the PEP-440 format is a strict subset of the Poetry format, isn't it? So as long as nobody uses caret or tilde requirements, the existing code should continue to work.

Maybe it doesn't make sense to maintain two separate parsers when 90% of the code is going to be duplicated.

Let me think about this a bit.

@mgedmin
Copy link
Owner

mgedmin commented Feb 7, 2023

Hmm, the PEP-440 format is a strict subset of the Poetry format.

Or maybe not. I don't see === or ~= in the Poetry spec.

@gpongelli
Copy link
Contributor

I've created a test marked with @pytest.mark.xfail in git master, specifically

@pytest.mark.xfail(
reason="Poetry-style Python requirements need implementation")
def test_get_python_requires_poetry(tmp_path, fix_max_python_3_version):
fix_max_python_3_version(9)
pyproject_toml = tmp_path / "pyproject.toml"
pyproject_toml.write_text(textwrap.dedent("""\
[tool.poetry.dependencies]
python = "^3.8"
"""))
assert get_python_requires(pyproject_toml) == v(['3.8', '3.9'])

it's not correct, on line 214 it should be ~ -> ~3.8 is equal to >=3.8.0,<3.9.0

caret version ^3.8 is equal to >=3.8.0,<4.0.0 .

Seems does not exist any library that parses poetry-like dependencies.

Start a branch with some tests added to verify poetry version constraints, I'll try to work there.

@gpongelli
Copy link
Contributor

I made some awful implementation that interprets tilde and caret, returning the same string expected as described on poetry's version constraint:

def last_zero_idx(lst: list) -> int:
    idx = 0
    for _i in range(0, len(lst)):
        if lst[_i] == '0':
            idx = _i
    return idx


def all_zero_list(lst: list) -> bool:
    ret = True
    for l in lst:
        if l and l != '0':
            ret = False
    return ret


if __name__ == '__main__':
    import re
    _r = re.compile(r'([~^])(\d)(.(\d)){0,1}(.(\d)){0,1}')

    _str = ['d', '^0', '^0.0', '^0.0.0', '^0.2.3', '^0.0.3', '^1', '^1.2', '^1.2.3', '~1', '~1.2', '~1.2.3']

    for _s in _str:
        _v = re.findall(_r, _s)
        if _v:
            _symbol = _v[0][0]
            _groups = _v[0]
            _filtered = list(map(lambda x: '0' if not x else x, _groups))
            print("------------\n" + _s)
            if _symbol == '^':
                _min_ver = '>=' + '.'.join(_filtered[1::2])
                _numbers = _groups[1::2]
                try:
                    _numbers.index('0')
                except ValueError as ve:
                    # 0 not in string
                    _mapped = list(map(lambda x: '0' if not x else x, _numbers))
                    _first_zero = 1
                    _mapped = _mapped[0:1] + ['0' for x in _mapped[1:]]
                    _mapped[_first_zero - 1] = str(int(_mapped[_first_zero - 1]) + 1)
                else:
                    _l = len(list(filter(lambda x: x, _numbers)))
                    _zero_list = all_zero_list(_numbers)
                    _last_zero_idx = last_zero_idx(_numbers)

                    _mapped = list(map(lambda x: '0' if not x else x, _numbers))
                    if _zero_list:
                        _mapped[_last_zero_idx] = '1'
                    else:
                        _mapped[_last_zero_idx + 1] = str(int(_mapped[_last_zero_idx + 1]) + 1)
                        for _i in range(_last_zero_idx + 2, len(_mapped)):
                            _mapped[_i] = '0'
                _max_ver = '<' + '.'.join(_mapped)
                print(f"{_min_ver} {_max_ver}")

            if _symbol == '~':
                _min_ver = '>=' + '.'.join(_filtered[1::2])
                _numbers = _groups[1::2]
                _mapped = list(map(lambda x: '0' if not x else x, _numbers))
                try:
                    _numbers.index('0')
                except ValueError as ve:
                    # 0 not in string
                    try:
                        _first_zero = _mapped.index('0')
                    except ValueError:
                        _first_zero = len(_mapped) -1

                    _mapped[_first_zero - 1] = str(int(_mapped[_first_zero - 1]) + 1)
                    _mapped[_first_zero] = '0'
                _max_ver = '<' + '.'.join(_mapped)
                print(f"{_min_ver} {_max_ver}")

I don't know if this can be integrated (probably the _min_ver or _max_ver can be parsed with Version) or how can be used, but here it is 😉

@mgedmin
Copy link
Owner

mgedmin commented Feb 8, 2023

it's not correct, on line 214 it should be ~ -> ~3.8 is equal to >=3.8.0,<3.9.0

caret version ^3.8 is equal to >=3.8.0,<4.0.0 .

The test uses ^3.8 and asserts that it is equivalent to >= 3.8, < 4.0, so it seems fine to me?

@mgedmin
Copy link
Owner

mgedmin commented Feb 8, 2023

One thing I'm not sure about Poetry version specs is if plain versions are allowed, e.g.

[tool.poetry.dependencies]
python = "3.7.10"
# or
python = "3.7"
# or
python = "3"

and if so, will they be treated the same as specifying ==3.7.10/==3.7/==3? (Some of these probably don't make sense.)

@mgedmin mgedmin closed this as completed in f3707f0 Feb 8, 2023
@mgedmin
Copy link
Owner

mgedmin commented Feb 8, 2023

I got nerd-sniped into implementing this. Please test and report bugs!

I myself tested it on https://github.com/jedie/python-creole, and there are some things I'm not entirely happy with, e.g.

$ check-python-versions --diff --drop 3.7 --only pyproject.toml 
--- ./pyproject.toml	(original)
+++ ./pyproject.toml	(updated)
@@ -11,13 +11,11 @@
 homepage = "https://github.com/jedie/python-creole/"
 keywords=["creole", "markup", "creole2html", "html2creole", "rest2html", "html2rest", "html2textile"]
 classifiers = [
-    # http://pypi.python.org/pypi?%3Aaction=list_classifiers
     "Development Status :: 5 - Production/Stable",
     "Environment :: Web Environment",
     "Intended Audience :: Developers",
     "License :: OSI Approved :: GNU General Public License (GPL)",
     "Programming Language :: Python",
-    "Programming Language :: Python :: 3.7",
     "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",

--- ./pyproject.toml	(original)
+++ ./pyproject.toml	(updated)
@@ -36,7 +36,7 @@
 include = ['AUTHORS', 'LICENSE']
 
 [tool.poetry.dependencies]
-python = ">=3.7,<4.0.0"
+python = ">=3.8"
 docutils = "*"
 
 [tool.poetry.dev-dependencies]

drops the comment from the classifier list and drops the <4.0.0 constraint. Both should be fixable, but I need to do some actual paid work today too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants