-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Implement --uploaded-prior-to to filter packages by upload-time
#13520
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 30 commits
e1e72e7
c8b2481
a53d08a
6db6c94
c0ec2ec
2bf1d3a
d5adbda
c374b2c
bc162b2
72f363d
181a7ca
007caf6
b53c5e8
0f1bc46
ad90024
6ff91a4
fbe923d
703cdc4
6cf2bec
4713c6d
841ae12
61ec9b0
e1f274a
e592e95
3cc912c
fc40558
afb6f2d
b5e4923
8be9e32
659d538
64f4529
a3b3ac1
b8f3513
4e3aecd
b6b6a7f
30fd4e1
f1831e7
50e45d2
9465e3e
8524919
5453030
0de1334
b9a21b5
97b6de1
707c449
c5e19c2
c655049
830324b
1d679b8
84c955b
8e595f7
b7ca0bf
6183877
9587d29
3f0350e
8c5db4d
b7e7ecf
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -257,6 +257,61 @@ e.g. http://example.com/constraints.txt, so that your organization can store and | |
| serve them in a centralized place. | ||
|
|
||
|
|
||
| .. _`Filtering by Upload Time`: | ||
|
|
||
|
|
||
| Filtering by Upload Time | ||
| ========================= | ||
|
|
||
| The ``--uploaded-prior-to`` option allows you to filter packages by their upload time | ||
| to an index, only considering packages that were uploaded before a specified datetime. | ||
| This can be useful for creating reproducible builds by ensuring you only install | ||
| packages that were available at a known point in time. | ||
|
|
||
| .. tab:: Unix/macOS | ||
|
|
||
| .. code-block:: shell | ||
|
|
||
| python -m pip install --uploaded-prior-to=2025-03-16T00:00:00Z SomePackage | ||
|
|
||
| .. tab:: Windows | ||
|
|
||
| .. code-block:: shell | ||
|
|
||
| py -m pip install --uploaded-prior-to=2025-03-16T00:00:00Z SomePackage | ||
|
|
||
| The option accepts ISO 8601 datetime strings in several formats: | ||
|
|
||
| * ``2025-03-16`` - Date in local timezone | ||
| * ``2025-03-16 12:30:00`` - Datetime in local timezone | ||
| * ``2025-03-16T12:30:00Z`` - Datetime in UTC | ||
| * ``2025-03-16T12:30:00+05:00`` - Datetime in UTC offset | ||
|
|
||
| For consistency across machines, use either UTC format (with 'Z' suffix) or UTC offset | ||
| format (with timezone offset like '+05:00'). Local timezone formats may produce different | ||
| results on different machines. | ||
|
|
||
| .. note:: | ||
|
|
||
| This option only works with package indexes that provide upload-time metadata | ||
| (such as PyPI). When upload-time information is not available, packages are not | ||
|
||
| filtered and installation continues normally. | ||
|
|
||
| You can combine this option with other filtering mechanisms like constraints files: | ||
|
|
||
| .. tab:: Unix/macOS | ||
|
|
||
| .. code-block:: shell | ||
|
|
||
| python -m pip install -c constraints.txt --uploaded-prior-to=2025-03-16 SomePackage | ||
|
|
||
| .. tab:: Windows | ||
|
|
||
| .. code-block:: shell | ||
|
|
||
| py -m pip install -c constraints.txt --uploaded-prior-to=2025-03-16 SomePackage | ||
|
|
||
|
|
||
| .. _`Dependency Groups`: | ||
|
|
||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| Add ``--uploaded-prior-to`` option to only consider packages uploaded prior to | ||
| a given datetime when the ``upload-time`` field is available from an index. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,14 @@ | ||
| from __future__ import annotations | ||
|
|
||
| import datetime | ||
| import functools | ||
| import itertools | ||
| import logging | ||
| import os | ||
| import posixpath | ||
| import re | ||
| import urllib.parse | ||
| import urllib.request | ||
| from collections.abc import Mapping | ||
| from dataclasses import dataclass | ||
| from typing import ( | ||
|
|
@@ -15,6 +17,7 @@ | |
| NamedTuple, | ||
| ) | ||
|
|
||
| from pip._internal.utils.datetime import parse_iso_datetime | ||
| from pip._internal.utils.deprecation import deprecated | ||
| from pip._internal.utils.filetypes import WHEEL_EXTENSION | ||
| from pip._internal.utils.hashes import Hashes | ||
|
|
@@ -207,6 +210,7 @@ class Link: | |
| "requires_python", | ||
| "yanked_reason", | ||
| "metadata_file_data", | ||
| "upload_time", | ||
| "cache_link_parsing", | ||
| "egg_fragment", | ||
| ] | ||
|
|
@@ -218,6 +222,7 @@ def __init__( | |
| requires_python: str | None = None, | ||
| yanked_reason: str | None = None, | ||
| metadata_file_data: MetadataFile | None = None, | ||
| upload_time: datetime.datetime | None = None, | ||
| cache_link_parsing: bool = True, | ||
| hashes: Mapping[str, str] | None = None, | ||
| ) -> None: | ||
|
|
@@ -239,6 +244,8 @@ def __init__( | |
| no such metadata is provided. This argument, if not None, indicates | ||
| that a separate metadata file exists, and also optionally supplies | ||
| hashes for that file. | ||
| :param upload_time: upload time of the file, or None if the information | ||
| is not available from the server. | ||
| :param cache_link_parsing: A flag that is used elsewhere to determine | ||
| whether resources retrieved from this link should be cached. PyPI | ||
| URLs should generally have this set to False, for example. | ||
|
|
@@ -272,6 +279,7 @@ def __init__( | |
| self.requires_python = requires_python if requires_python else None | ||
| self.yanked_reason = yanked_reason | ||
| self.metadata_file_data = metadata_file_data | ||
| self.upload_time = upload_time | ||
|
|
||
| self.cache_link_parsing = cache_link_parsing | ||
| self.egg_fragment = self._egg_fragment() | ||
|
|
@@ -300,6 +308,12 @@ def from_json( | |
| if metadata_info is None: | ||
| metadata_info = file_data.get("dist-info-metadata") | ||
|
|
||
| upload_time: datetime.datetime | None | ||
| if upload_time_data := file_data.get("upload-time"): | ||
| upload_time = parse_iso_datetime(upload_time_data) | ||
| else: | ||
| upload_time = None | ||
|
Comment on lines
+311
to
+315
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I know that the JSON version of the Simple API is used in the vast majority of installs, but we should probably support this feature with the HTML Simple API if possible. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This feature doesn't exist in the HTML version of the API, it only exists as addition JSON fields in the spec: https://peps.python.org/pep-0700/#specification
IMO this was a short sighted choice by the spec authors, as it has prevented certain simple index libraries from supporting this feature. But we can't change the past. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wow, thanks for the information. /me wonders if it'd be useful to propose a v1.1 of the HTML spec to bring it to feature parity with the JSON API, but I don't have the time for that, haha. |
||
|
|
||
| # The metadata info value may be a boolean, or a dict of hashes. | ||
| if isinstance(metadata_info, dict): | ||
| # The file exists, and hashes have been supplied | ||
|
|
@@ -325,6 +339,7 @@ def from_json( | |
| yanked_reason=yanked_reason, | ||
| hashes=hashes, | ||
| metadata_file_data=metadata_file_data, | ||
| upload_time=upload_time, | ||
| ) | ||
|
|
||
| @classmethod | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In
pypi-timemachineall date/times are specified in UTC. It is hard to imagine a use case where using anything else is a good idea.Is this a case of following the ISO standard in
pipis harmful to inexperienced users?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was previously discussed: #13520 (comment)