feat!: add Python 3.12 support and drop Python 3.8 support (#335)

This change coincides with the release of Python 3.12, which happened
02 OCT 2023. Anyone tracking this repository and this change should
update their use of pyenv locally, to include py3.12.x

Changes made to add Python 3.12 support include:

* Update the Docker base image version
* Update the CONTRIBUTING documentation
* Update the testing environments in `tox`, workflows, and unit tests
* Update `pyproject.toml` and generate an updated `poetry.lock` lockfile
  * The existing `poetry-core` specified minimum requirement still works

Changes made to remove Python 3.8 support include:

* Update `pyproject.toml` and regenerate `poetry.lock`
  * Upgrade target version for QA tools
* Use the new `str.removeprefix` and `str.removesuffix` methods
* Use `pyupgrade` to automatically update code
* Add a missing pre-requisite in the `CIPreCommit` implementation
  * Checking `self.lockfiles` requires project settings to be available
  * Add a call to `self._backup_project_file()`
  * Change `_backup_project_file()` to be cached so it only runs once
    * This is preferred over moving the call to the function in `CIBase`
      up during initialization, to be before `_check_prerequisites()`,
      since none of the other implementations require it
* Update tests
  * Remove Python 3.8 from `tox` config
  * Update the test workflow
  * Update the test for supported Python versions
* Update documentation and remove references to Python 3.8
* Fix typos

Closes: #320
Closes: #215

BREAKING CHANGE: Support for Python 3.8 was removed due to the change in
CONTRIBUTING policy to support only the current/latest release plus the
previous three minor versions of Python.

.github/workflows/auto_updates.yml

@@ -17,7 +17,7 @@ jobs:
     strategy:
       matrix:
         # It's only one Python version specified in a "matrix", but on purpose to stay DRY
-        python-version: ["3.11"]
+        python-version: ["3.12"]
     defaults:
       run:
         shell: bash

.github/workflows/preview.yml

@@ -25,7 +25,7 @@ jobs:
     strategy:
       matrix:
         # It's only one Python version specified in a "matrix", but on purpose to stay DRY
-        python-version: ["3.11"]
+        python-version: ["3.12"]
     defaults:
       run:
         shell: bash

.github/workflows/release.yml

@@ -38,7 +38,7 @@ jobs:
     strategy:
       matrix:
         # It's only one Python version specified in a "matrix", but on purpose to stay DRY
-        python-version: ["3.11"]
+        python-version: ["3.12"]
     defaults:
       run:
         shell: bash

.github/workflows/test.yml

@@ -21,7 +21,7 @@ jobs:
     strategy:
       matrix:
         # It's only one Python version specified in a "matrix", but on purpose to stay DRY
-        python-version: ["3.11"]
+        python-version: ["3.12"]
     steps:
       - name: Checkout the repo
         uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
@@ -64,7 +64,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.8", "3.9", "3.10", "3.11"]
+        python-version: ["3.9", "3.10", "3.12"]
     steps:
       - name: Checkout the repo
         uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
@@ -101,7 +101,7 @@ jobs:
       fail-fast: false
       matrix:
         # It's only one Python version specified in a "matrix", but on purpose to stay DRY
-        python-version: ["3.11"]
+        python-version: ["3.12"]
         dockerfile: ["Dockerfile", "Dockerfile.slim"]
         build: ["wheel", "source"]
     env:

CONTRIBUTING.md

@@ -102,9 +102,9 @@ Here's how to set up `phylum-ci` for local development.
     ```
 
 3. Ensure all supported Python versions are installed locally
-   1. The strategy is to support all released minor versions of Python that are not end-of-life yet
+   1. The strategy is to support the current/latest release plus the previous three minor versions of Python
    2. The current list
-      1. at the time of this writing is 3.8, 3.9, 3.10, and 3.11
+      1. at the time of this writing is 3.9, 3.10, 3.11, and 3.12
       2. can be inferred with the Python Developer's Guide, which maintains the
          [status of active Python releases](https://devguide.python.org/versions/)
    3. It is recommended to use [`pyenv`](https://github.com/pyenv/pyenv) to manage multiple Python installations
@@ -116,14 +116,14 @@ Here's how to set up `phylum-ci` for local development.
 
     # NOTE: These versions are examples; the latest patch version available from
     #       pyenv should be used in place of `.x`.
-    pyenv install 3.8.x
     pyenv install 3.9.x
     pyenv install 3.10.x
     pyenv install 3.11.x
+    pyenv install 3.12.x
     pyenv rehash
 
     # Ensure all environments are available globally (helps tox to find them)
-    pyenv global 3.11.x 3.10.x 3.9.x 3.8.x
+    pyenv global 3.12.x 3.11.x 3.10.x 3.9.x
     ```
 
 4. Ensure [poetry v1.6+](https://python-poetry.org/docs/) is installed
@@ -216,7 +216,7 @@ Before you submit a pull request, check that it meets these guidelines:
 * Have you created sufficient tests?
 * Have you updated all affected documentation?
 
-The pull request should work for Python 3.8, 3.9, 3.10, and 3.11.
+The pull request should work for Python 3.9, 3.10, 3.11, and 3.12.
 Check <https://github.com/phylum-dev/phylum-ci/actions> and make sure that the tests
 pass for all supported Python versions.
 

Dockerfile

@@ -69,7 +69,7 @@
 # $ scripts/docker_tests.sh --image phylum-ci
 ##########################################################################################
 
-FROM python:3.11-slim-bookworm AS builder
+FROM python:3.12-slim-bookworm AS builder
 
 # PKG_SRC is the path to a built distribution/wheel and PKG_NAME is the name of the built
 # distribution/wheel. Both can optionally be specified in glob form. When not defined,
@@ -121,7 +121,7 @@ RUN find ${PHYLUM_VENV} -type f -name '*.pyc' -delete
 # in the final layer and also known to be part of the $PATH
 COPY entrypoint.sh ${PHYLUM_VENV}/bin/
 
-FROM python:3.11-slim-bookworm
+FROM python:3.12-slim-bookworm
 
 # CLI_VER specifies the Phylum CLI version to install in the image.
 # Values should be provided in a format acceptable to the `phylum-init` script.

Dockerfile.slim

@@ -77,7 +77,7 @@
 # $ scripts/docker_tests.sh --image phylum-ci --slim
 ##########################################################################################
 
-FROM python:3.11-slim-bookworm AS builder
+FROM python:3.12-slim-bookworm AS builder
 
 # PKG_SRC is the path to a built distribution/wheel and PKG_NAME is the name of the built
 # distribution/wheel. Both can optionally be specified in glob form. When not defined,
@@ -129,7 +129,7 @@ RUN find ${PHYLUM_VENV} -type f -name '*.pyc' -delete
 # in the final layer and also known to be part of the $PATH
 COPY entrypoint.sh ${PHYLUM_VENV}/bin/
 
-FROM python:3.11-slim-bookworm
+FROM python:3.12-slim-bookworm
 
 # CLI_VER specifies the Phylum CLI version to install in the image.
 # Values should be provided in a format acceptable to the `phylum-init` script.

README.md

@@ -47,7 +47,7 @@ pipx run --spec phylum phylum-init <options>
 pipx run --spec phylum phylum-ci <options>
 ```
 
-These installation methods require Python 3.8+ to run.
+These installation methods require Python 3.9+ to run.
 For a self contained environment, consider using the Docker image as described below.
 
 ### Usage

pyproject.toml

@@ -27,10 +27,10 @@ classifiers = [
     "Topic :: Software Development",
     "Topic :: Software Development :: Quality Assurance",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
 ]
 packages = [
     { include = "phylum", from = "src" },
@@ -46,7 +46,7 @@ phylum-init = "phylum.init.cli:main"
 phylum-ci = "phylum.ci.cli:script_main"
 
 [tool.poetry.dependencies]
-python = ">=3.8,<3.12"
+python = ">=3.9,<3.13"
 requests = "*"
 cryptography = "*"
 packaging = "*"
@@ -90,13 +90,13 @@ paths = ["src", "tests"]
 exclude = ["conftest.py"]
 
 [tool.refurb]
-python_version = "3.8"
+python_version = "3.9"
 format = "github"
 
 [tool.ruff]
 # Reference: https://beta.ruff.rs/docs/settings
 line-length = 120
-target-version = "py38"
+target-version = "py39"
 force-exclude = true
 src = ["src", "tests"]
 select = [
@@ -119,7 +119,7 @@ ignore = [
     "B019", # cached-instance-method
     # Assigning to a variable before a return statement is more readable and useful for debugging
     "RET504", # unnecessary-assign
-    # Allowing exception handling within loops improves readability with ony a negligible performance impact.
+    # Allowing exception handling within loops improves readability with only a negligible performance impact.
     "PERF203", # try-except-in-loop
     # These ignores will be removed during https://github.com/phylum-dev/phylum-ci/issues/238
     "ANN", # flake8-annotations

src/phylum/ci/ci_azure.py

@@ -23,7 +23,7 @@
 import shlex
 import subprocess
 import textwrap
-from typing import Dict, List, Optional, Tuple
+from typing import Optional
 import urllib.parse
 
 import requests
@@ -152,11 +152,7 @@ def phylum_label(self) -> str:
             pr_number = os.getenv("SYSTEM_PULLREQUEST_PULLREQUESTNUMBER")
             if pr_number is None:
                 pr_number = os.getenv("SYSTEM_PULLREQUEST_PULLREQUESTID", "unknown-number")
-            pr_src_branch = os.getenv("SYSTEM_PULLREQUEST_SOURCEBRANCH", "unknown-ref")
-            ref_prefix = "refs/heads/"
-            # Starting with Python 3.9, the str.removeprefix() method was introduced to do this same thing
-            if pr_src_branch.startswith(ref_prefix):
-                pr_src_branch = pr_src_branch.replace(ref_prefix, "", 1)
+            pr_src_branch = os.getenv("SYSTEM_PULLREQUEST_SOURCEBRANCH", "unknown-ref").removeprefix("refs/heads/")
             label = f"{self.ci_platform_name}_PR#{pr_number}_{pr_src_branch}"
         else:
             current_branch = os.getenv("BUILD_SOURCEBRANCHNAME", "unknown-branch")
@@ -279,7 +275,7 @@ def post_output(self) -> None:
             post_github_comment(comments_url, self.github_token, self.analysis_report)
 
 
-def get_pr_branches() -> Tuple[str, str]:
+def get_pr_branches() -> tuple[str, str]:
     """Get the source and destination branches when in a PR context and return them as a tuple."""
     # There is no single predefined variable available to provide the PR base SHA.
     # Instead, it can be determined with a `git merge-base` command, like is done for the CINone implementation.
@@ -325,7 +321,7 @@ def get_most_recent_phylum_comment_azure(azure_token: str) -> Optional[str]:
     if resp.status_code != requests.codes.OK:
         msg = f"Are the permissions on the Azure token `AZURE_TOKEN` correct? {AZURE_PAT_ERR_MSG}"
         raise SystemExit(msg)
-    pr_threads_resp: Dict = resp.json()
+    pr_threads_resp: dict = resp.json()
 
     pr_threads_count = pr_threads_resp.get("count", 0)
     LOG.debug("PR threads found: %s", pr_threads_count)
@@ -334,14 +330,14 @@ def get_most_recent_phylum_comment_azure(azure_token: str) -> Optional[str]:
         return None
 
     LOG.info("Checking pull request threads for existing Phylum-generated comments ...")
-    pr_threads: List = pr_threads_resp.get("value", [])
+    pr_threads: list = pr_threads_resp.get("value", [])
     # NOTE: The API call returns the comments in ascending order by ID...thus the need to reverse the list.
     #       Detecting Phylum comments is done simply by looking for those that start with a known string value.
     #       We only care about the most recent Phylum comment.
-    pr_thread: Dict
+    pr_thread: dict
     for pr_thread in reversed(pr_threads):
         thread_comments = pr_thread.get("comments", [])
-        thread_comment: Dict
+        thread_comment: dict
         for thread_comment in thread_comments:
             # All Phylum generated comments will be the first in their own thread
             if thread_comment.get("id", 0) != 1:
@@ -396,7 +392,7 @@ def post_azure_comment(azure_token: str, comment: str) -> None:
     resp.raise_for_status()
 
 
-def get_headers(azure_token: str) -> Dict[str, str]:
+def get_headers(azure_token: str) -> dict[str, str]:
     """Provide the headers to use when making Azure Pipelines API calls."""
     # To provide the personal access token through an HTTP header, you must first convert it to a base64 string.
     # NOTE: The colon (`:`) appended to the front of the PAT is intentional as it is expected by the endpoints.
@@ -408,7 +404,7 @@ def get_headers(azure_token: str) -> Dict[str, str]:
     return headers
 
 
-def get_query_params() -> Tuple[Dict, str]:
+def get_query_params() -> tuple[dict, str]:
     """Provide the query parameters to use when making Azure Pipelines API calls."""
     # This is the latest available API version a/o SEP 2023. While it is a "preview" version, it was chosen to
     # lean forward in an effort to maintain relevance and recency for a longer period of time going forward.