Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recursively purge all download URLs #162

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 19 additions & 15 deletions run_release.py
Original file line number Diff line number Diff line change
Expand Up @@ -942,22 +942,26 @@
"https://www.python.org/downloads/windows/",
"https://www.python.org/downloads/macos/",
]
# Purge the source URLs and their associated metadata files.
source_urls = [
f"https://www.python.org/ftp/python/{normalized_release}/Python-{db['release']}.tgz",
f"https://www.python.org/ftp/python/{normalized_release}/Python-{db['release']}.tar.xz",
]
for source_url in source_urls:
urls.extend(
[
f"{source_url}",
f"{source_url}.asc",
f"{source_url}.crt",
f"{source_url}.sig",
f"{source_url}.sigstore",
f"{source_url}.spdx.json",
]

# Recursively discover artifacts to purge.
ftp_download_pages = [f"https://www.python.org/ftp/python/{normalized_release}/"]
while ftp_download_pages:
ftp_download_page = ftp_download_pages.pop(0)
req = urllib.request.Request(

Check warning on line 950 in run_release.py

View check run for this annotation

Codecov / codecov/patch

run_release.py#L947-L950

Added lines #L947 - L950 were not covered by tests
method="GET", url=ftp_download_page, headers=headers
)
resp = urllib.request.urlopen(req)
if resp.code != 200:
raise RuntimeError("Failed to purge the python.org/downloads CDN")
for link in re.findall(r"<a href=\"([^\"]+)\">", resp.read().decode()):
if link in ("../", "./"): # Special value, ignore it.
continue

Check warning on line 958 in run_release.py

View check run for this annotation

Codecov / codecov/patch

run_release.py#L953-L958

Added lines #L953 - L958 were not covered by tests

if link.endswith("/"): # Directory, recurse into it.
ftp_download_pages.append(f"{ftp_download_page}{link}")

Check warning on line 961 in run_release.py

View check run for this annotation

Codecov / codecov/patch

run_release.py#L960-L961

Added lines #L960 - L961 were not covered by tests

# We want to purge both directories and files.
urls.append(f"{ftp_download_page}{link}")

Check warning on line 964 in run_release.py

View check run for this annotation

Codecov / codecov/patch

run_release.py#L964

Added line #L964 was not covered by tests

for url in urls:
req = urllib.request.Request(url=url, headers=headers, method="PURGE")
Expand Down