Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Python3.13 Support and housekeeping #4276

Closed
wants to merge 10 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/all-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
strategy:
matrix:
os: [ windows-latest, ubuntu-latest ]
python-version: [ "3.9", "3.10", "3.11", "3.12" ]
python-version: [ "3.9", "3.10", "3.11", "3.12", "3.13" ]
uses: ./.github/workflows/unit-tests.yml
with:
os: ${{ matrix.os }}
Expand All @@ -36,7 +36,7 @@ jobs:
strategy:
matrix:
os: [ windows-latest, ubuntu-latest ]
python-version: [ "3.9", "3.10", "3.11", "3.12" ]
python-version: [ "3.9", "3.10", "3.11", "3.12", "3.13" ]
uses: ./.github/workflows/e2e-tests.yml
with:
os: ${{ matrix.os }}
Expand All @@ -59,7 +59,7 @@ jobs:
strategy:
matrix:
os: [ windows-latest, ubuntu-latest ]
python-version: [ "3.9", "3.10", "3.11", "3.12" ]
python-version: [ "3.9", "3.10", "3.11", "3.12", "3.13" ]
uses: ./.github/workflows/pip-compile.yml
with:
os: ${{ matrix.os }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/check-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
python-version: '3.11'
- name: Install uv
run: |
python -m pip install "uv==0.1.32"
python -m pip install "uv==0.4.29"
- name: Install dependencies
run: |
uv pip install --system requests
Expand Down Expand Up @@ -52,7 +52,7 @@ jobs:
python-version: '3.11'
- name: Install uv
run: |
python -m pip install "uv==0.1.32"
python -m pip install "uv==0.4.29"
- name: Install dependencies
run: |
uv pip install --system build
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/docs-only-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
strategy:
matrix:
os: [ ubuntu-latest ]
python-version: [ "3.9", "3.10", "3.11", "3.12" ]
python-version: [ "3.9", "3.10", "3.11", "3.12", "3.13" ]
uses: ./.github/workflows/lint.yml
with:
os: ${{ matrix.os }}
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ default_stages: [pre-commit, manual]

repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.6.1
rev: v0.7.1
hooks:
- id: ruff
name: "ruff on kedro/, tests/ and docs/"
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ package: clean install
python -m pip install build && python -m build

install-test-requirements:
python -m pip install "uv==0.1.32"
python -m pip install "uv==0.4.29"
uv pip install --system "kedro[test] @ ."

install-pre-commit:
Expand Down
1 change: 1 addition & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## Major features and improvements
* Implemented dict-like interface for `KedroDataCatalog`.
* Added Python 3.13 support.

**Note:** ``KedroDataCatalog`` is an experimental feature and is under active development. Therefore, it is possible we'll introduce breaking changes to this class, so be mindful of that if you decide to use it already. Let us know if you have any feedback about the ``KedroDataCatalog`` or ideas for new features.

Expand Down
2 changes: 1 addition & 1 deletion kedro/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ class KedroPythonVersionWarning(UserWarning):
warnings.simplefilter("default", KedroDeprecationWarning)
warnings.simplefilter("error", KedroPythonVersionWarning)

if sys.version_info >= (3, 13):
if sys.version_info >= (3, 14):
warnings.warn(
"""Kedro is not yet fully compatible with this Python version.
To proceed at your own risk and ignore this warning,
Expand Down
120 changes: 62 additions & 58 deletions kedro/framework/cli/micropkg.py
Original file line number Diff line number Diff line change
Expand Up @@ -211,67 +211,71 @@ def _pull_package( # noqa: PLR0913
destination: str | None = None,
fs_args: str | None = None,
) -> None:
with tempfile.TemporaryDirectory() as temp_dir:
temp_dir_path = Path(temp_dir).resolve()
_unpack_sdist(package_path, temp_dir_path, fs_args)

# temp_dir_path is the parent directory of the project root dir
contents = [member for member in temp_dir_path.iterdir() if member.is_dir()]
if len(contents) != 1:
raise KedroCliError(
"Invalid sdist was extracted: exactly one directory was expected, "
f"got {contents}"
)
project_root_dir = contents[0]

# This is much slower than parsing the requirements
# directly from the metadata files
# because it installs the package in an isolated environment,
# but it's the only reliable way of doing it
# without making assumptions on the project metadata.
library_meta = project_wheel_metadata(project_root_dir)

# Project name will be `my-pipeline` even if `pyproject.toml` says `my_pipeline`
# because standards mandate normalization of names for comparison,
# see https://packaging.python.org/en/latest/specifications/core-metadata/#name
# The proper way to get it would be
# project_name = library_meta.get("Name")
# However, the rest of the code expects the non-normalized package name,
# so we have to find it.
packages = [
project_item.name
for project_item in project_root_dir.iterdir()
if project_item.is_dir()
and project_item.name != "tests"
and (project_item / "__init__.py").exists()
]
if len(packages) != 1:
# Should not happen if user is calling `micropkg pull`
# with the result of a `micropkg package`,
# and in general if the distribution only contains one package (most likely),
# but helps give a sensible error message otherwise
raise KedroCliError(
"Invalid package contents: exactly one package was expected, "
f"got {packages}"
)
package_name = packages[0]
import tempfile

temp_dir = tempfile.TemporaryDirectory()
temp_dir_path = Path(temp_dir).resolve()
_unpack_sdist(package_path, temp_dir_path, fs_args)

# temp_dir_path is the parent directory of the project root dir
contents = [member for member in temp_dir_path.iterdir() if member.is_dir()]
if len(contents) != 1:
raise KedroCliError(
"Invalid sdist was extracted: exactly one directory was expected, "
f"got {contents}"
)
project_root_dir = contents[0]

# This is much slower than parsing the requirements
# directly from the metadata files
# because it installs the package in an isolated environment,
# but it's the only reliable way of doing it
# without making assumptions on the project metadata.
library_meta = project_wheel_metadata(project_root_dir)

# Project name will be `my-pipeline` even if `pyproject.toml` says `my_pipeline`
# because standards mandate normalization of names for comparison,
# see https://packaging.python.org/en/latest/specifications/core-metadata/#name
# The proper way to get it would be
# project_name = library_meta.get("Name")
# However, the rest of the code expects the non-normalized package name,
# so we have to find it.
packages = [
project_item.name
for project_item in project_root_dir.iterdir()
if project_item.is_dir()
and project_item.name != "tests"
and (project_item / "__init__.py").exists()
]
if len(packages) != 1:
# Should not happen if user is calling `micropkg pull`
# with the result of a `micropkg package`,
# and in general if the distribution only contains one package (most likely),
# but helps give a sensible error message otherwise
raise KedroCliError(
"Invalid package contents: exactly one package was expected, "
f"got {packages}"
)
package_name = packages[0]

# Type ignored because of https://github.com/pypa/build/pull/693
package_reqs = _get_all_library_reqs(library_meta) # type: ignore[arg-type]
# Type ignored because of https://github.com/pypa/build/pull/693
package_reqs = _get_all_library_reqs(library_meta) # type: ignore[arg-type]

if package_reqs:
requirements_txt = metadata.project_path / "requirements.txt"
_append_package_reqs(requirements_txt, package_reqs, package_name)
if package_reqs:
requirements_txt = metadata.project_path / "requirements.txt"
_append_package_reqs(requirements_txt, package_reqs, package_name)

_clean_pycache(temp_dir_path)
_install_files(
metadata,
package_name,
project_root_dir,
env,
alias,
destination,
)
_clean_pycache(temp_dir_path)
_install_files(
metadata,
package_name,
project_root_dir,
env,
alias,
destination,
)
temp_dir.cleanup()
click.secho("dummy")


def _pull_packages_from_manifest(metadata: ProjectMetadata) -> None:
Expand Down
8 changes: 4 additions & 4 deletions tests/test_import.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@


def test_import_kedro_with_no_official_support_raise_error(mocker):
"""Test importing kedro with python>=3.13 should fail"""
mocker.patch("kedro.sys.version_info", (3, 13))
"""Test importing kedro with python>=3.14 should fail"""
mocker.patch("kedro.sys.version_info", (3, 14))

# We use the parent class to avoid issues with `exec_module`
with pytest.raises(UserWarning) as excinfo:
Expand All @@ -15,8 +15,8 @@ def test_import_kedro_with_no_official_support_raise_error(mocker):


def test_import_kedro_with_no_official_support_emits_warning(mocker):
"""Test importing kedro python>=3.13 and controlled warnings should work"""
mocker.patch("kedro.sys.version_info", (3, 13))
"""Test importing kedro python>=3.14 and controlled warnings should work"""
mocker.patch("kedro.sys.version_info", (3, 14))
mocker.patch("kedro.sys.warnoptions", ["default:Kedro is not yet fully compatible"])

# We use the parent class to avoid issues with `exec_module`
Expand Down