Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PEP 723 support #12891

Open
1 task done
SnoopJ opened this issue Aug 3, 2024 · 6 comments · May be fixed by #13052
Open
1 task done

PEP 723 support #12891

SnoopJ opened this issue Aug 3, 2024 · 6 comments · May be fixed by #13052
Labels
state: needs discussion This needs some more discussion type: feature request Request for a new feature

Comments

@SnoopJ
Copy link
Contributor

SnoopJ commented Aug 3, 2024 •
edited
Loading

What's the problem this feature will solve?

This feature request represents support for installing the requirements of a single-file script with inline metadata (PEP 723) using pip.

Pessimistically assuming that this feature request is rejected, this issue is also being filed as a home for a specific record of the maintainer rationale for not including this functionality in pip.

Describe the solution you'd like

I am imagining functionality that would be similar to the existing --requirement argument, with a suitably distinct name to indicate that the metadata format is the one defined by PEP 723 and the living PyPA spec.

Perhaps something like:

$ cat standalone_script.py
# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "requests<3",
#   "rich",
# ]
# ///

import requests
from rich.pretty import pprint

resp = requests.get("https://peps.python.org/api/peps.json")
data = resp.json()
pprint([(k, v["title"]) for k, v in data.items()][:10])

$ pip install --script standalone_script.py
...

Where the pip install invocation is equivalent to pip install "requests<3" "rich" if the requires-python bound is satisfied

Alternative Solutions

I know that other tools¹ in the Python packaging ecosystem do support PEP 723, but these tools are not "included batteries" nearly as often as pip is. In my estimation, supporting in-line metadata in pip will improve the experience of many users (especially non-programmers) who just want to run a script they've been given and are starting with a "typical" bare Python environment.

¹non-exhaustive list: pipx, hatch, pip-run

Additional context

The reference implementation for parsing inline metadata seems to be enough to retrieve the list of requirements from a file. I imagine that the result of this can be plumbed into existing machinery used for satisfying dependencies listed in requirements files.

It's worth mentioning that there is a potential argument for allowing multiple scripts to be parsed and installed from in a single pip invocation (in the same way that --requirement can be given multiple times). I think it makes sense but I can see why there might be some reluctance about doing it since it is more or less directly against the grain of the primary "single script" use-case of inline metadata.

Code of Conduct
@SnoopJ SnoopJ added S: needs triage Issues/PRs that need to be triaged type: feature request Request for a new feature labels Aug 3, 2024
@pfmoore
Copy link
Member

pfmoore commented Aug 3, 2024

I feel that this is a reasonable request, although personally I consider it to be a suboptimal use of inline script metadata - the point of that data is for a tool that runs the script to transparently manage the environment the script runs in. Manually managing the environment with pip is not the intended use and I wouldn’t want to see it become a common pattern.

Having said that, if someone wants to submit a PR, I wouldn’t object. It’s not a feature I intend to work on myself, though.

@ichard26 ichard26 added state: needs discussion This needs some more discussion and removed S: needs triage Issues/PRs that need to be triaged labels Aug 9, 2024
@vovavili
Copy link

vovavili commented Aug 16, 2024 •
edited
Loading

I feel that this is a reasonable request, although personally I consider it to be a suboptimal use of inline script metadata - the point of that data is for a tool that runs the script to transparently manage the environment the script runs in. Manually managing the environment with pip is not the intended use and I wouldn’t want to see it become a common pattern.

Having said that, if someone wants to submit a PR, I wouldn’t object. It’s not a feature I intend to work on myself, though.

I'm willing to dabble into this.
Would it be possible to just shift the solution from pipx? If so, a pull request should be quite easy.

@pfmoore
Copy link
Member

pfmoore commented Aug 16, 2024

I'm not sure what part of the pipx code you're thinking of. Parsing the PEP 723 metadata uses the code from the PEP itself, I believe, so it'll be the same code wherever you take it from. Most of the rest of the pipx code is about managing virtual environments, which is not suitable for pip.

The "fun" bit of this would likely be around designing the UI. Would the new --script-requirements (or whatever you call it) argument be allowed with --requirements, or are they mutually exclusive? Should --constraints be allowed? What do we do about a requires-python in the script requirements? Fail with an error if the target environment uses a Python version that doesn't match the constraint? How would that work with --target, where we don't know what Python version will be used?

I don't want to over-complicate the problem here, but I'd strongly encourage you to think about the design, rather than hoping to just copy/paste some code from another project. A bit of test-driven design, where you start by writing some tests to exercise the behaviour you want, and then write code to implement it (and hence "fix" the tests) would be very useful here, IMO.

@SnoopJ
Copy link
Contributor Author

SnoopJ commented Oct 5, 2024

Thinking about this issue again. I agree with Paul that the venv part of this is not pip's problem, thought I'd rattle off my personal answers to the explicit (but non-exhaustive) questions:

Would the new --script-requirements (or whatever you call it) argument be allowed with --requirements, or are they mutually exclusive? Should --constraints be allowed?

I don't really see any reason to exclude --requirements or --constraints, but I would want to avoid using the word "requirements" in the new functionality to avoid conflating the two distinct formats. I'm not heavily attached to the --script argument in the initial filing.

What do we do about a requires-python in the script requirements? Fail with an error if the target environment uses a Python version that doesn't match the constraint?

That's the way I read the PEP. It's not quite a requirement, but it does say that this SHOULD happen. I can't think of any reason to go against the recommendation, aside from…

How would that work with --target, where we don't know what Python version will be used?

That's an interesting question. I can see a case for ignoring requires-python in this scenario and issuing a warning if the Python running pip is known to not be compatible.

Would it be possible to just pypa/pipx#1100 the solution from pipx? If so, a pull request should be quite easy.

I'd strongly encourage you to think about the design, rather than hoping to just copy/paste some code from another project

+1, I think implementing this is going to require the kind of motivated design from where pip currently is that Paul is talking about. There's just too much difference with pipx in terms of what the tools do. In particular, pip is not and IMO should not become a script runner.

@SnoopJ
Copy link
Contributor Author

SnoopJ commented Oct 5, 2024 •
edited
Loading

Making a note here that the implications of this feature for pip download and pip wheel (the other commands that accept --requirements) should also be considered when drafting a design for this feature. I am personally most interested in the behavior for pip install but it makes sense to track the capabilities of --requirements as closely as is practical.

Edit: from a minimal draft I spent the last few hours putting together, it looks like functionality for those commands is "free" inasmuch as the commands that build requirement sets all get this functionality via req_command.py

@SnoopJ SnoopJ linked a pull request Oct 28, 2024 that will close this issue
Open
@clbarnes
Copy link

clbarnes commented Dec 3, 2024

A use case for this which I've come across is AWS lambdas where you want to install your dependencies to a specific folder so that you can then zip it up with your script. Allowing pip to read the dependencies from the inline script metadata means you don't need an extra requirements file, and that you can separate your runtime dependencies from your development dependencies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
state: needs discussion This needs some more discussion type: feature request Request for a new feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support installing requirements from inline script metadata (PEP 723) SnoopJ/pip
5 participants
@pfmoore @SnoopJ @clbarnes @ichard26 @vovavili