Skip to content

Latest commit

 

History

History
394 lines (346 loc) · 12.9 KB

README.md

File metadata and controls

394 lines (346 loc) · 12.9 KB
Raw

unpy

Unified Python

Transpiles .pyi stubs from Python 3.13 to 3.10

unpy - PyPI unpy - Python Versions unpy - license

unpy - CI unpy - pre-commit unpy - basedmypy unpy - basedpyright unpy - ruff

Important

This project is in the alpha stage: You probably shouldn't use it in production.

Installation

$ pip install unpy

Usage

$ unpy --help
Usage: unpy [OPTIONS] SOURCE [OUTPUT]

Arguments:
  SOURCE    Path to the input .pyi file or '-' to read from stdin.  [required]
  [OUTPUT]  Path to the output .pyi file. Defaults to stdout.

Options:
  --version                       Show the version and exit
  --diff                          Show the changes between the input and
                                  output in unified diff format
  --target [3.10|3.11|3.12|3.13]  The minimum Python version that should be
                                  supported.  [default: 3.10]
  --help                          Show this message and exit.

Examples

Some simple examples of Python 3.13 stubs that are backported to Python 3.10.

Imports

$ unpy --target 3.10 --diff examples/imports.pyi
+++ -
@@ -1,6 +1,4 @@
- from types import CapsuleType
- from typing import override
- from warnings import deprecated
+ from typing_extensions import CapsuleType, deprecated, override

  @deprecated("RTFM")
  class Spam:
      __pyx_capi__: dict[str, CapsuleType]
      @override
      def __hash__(self, /) -> int: ...

Note the alphabetical order of the generated imports.

Type Aliases

$ unpy --target 3.10 --diff examples/type_aliases.pyi
+++ -
@@ -1,7 +1,15 @@
  from collections.abc import Callable
+ from typing import ParamSpec, TypeAlias, TypeVar
+ from typing_extensions import TypeAliasType, TypeVarTuple, Unpack

- type Binary = bytes | bytearray | memoryview
- type Vector[R: float] = tuple[R, ...]
- type tciD[V, K] = dict[K, V]
- type Things[*Ts] = tuple[*Ts]
- type Callback[**Tss] = Callable[Tss, None]
+ _R = TypeVar("_R", bound=float)
+ _V = TypeVar("_V")
+ _K = TypeVar("_K")
+ _Ts = TypeVarTuple("_Ts")
+ _Tss = ParamSpec("_Tss")
+
+ Binary: TypeAlias = bytes | bytearray | memoryview
+ Vector: TypeAlias = tuple[_R, ...]
+ tciD = TypeAliasType("tciD", dict[_K, _V], type_params=(_V, _K))
+ Things: TypeAlias = tuple[Unpack[_Ts]]
+ Callback: TypeAlias = Callable[_Tss, None]

Note that TypeAlias cannot be used with tciD because the definition order of the type parameters (at the left-hand side) does not match the order in which they are accessed (at the right-hand side), and the backported TypeAliasType must be used instead.

Functions

$ unpy --target 3.10 --diff examples/functions.pyi
+++ -
@@ -1,6 +1,11 @@
+ _T = TypeVar("_T")
+ _S = TypeVar("_S", str, bytes)
+ _X = TypeVar("_X")
+ _Theta = ParamSpec("_Theta")
+ _Y = TypeVar("_Y")
  from collections.abc import Callable as Def
- from typing import Concatenate as Concat
+ from typing import Concatenate as Concat, ParamSpec, TypeVar

- def noop[T](x: T, /) -> T: ...
- def concat[S: (str, bytes)](left: S, right: S) -> S: ...
- def curry[X, **Theta, Y](f: Def[Concat[X, Theta], Y], /) -> Def[[X], Def[Theta, Y]]: ...
+ def noop(x: _T, /) -> _T: ...
+ def concat(left: _S, right: _S) -> _S: ...
+ def curry(f: Def[Concat[_X, _Theta], _Y], /) -> Def[[_X], Def[_Theta, _Y]]: ...

Generic classes and protocols

$ unpy --target 3.10 --diff examples/generics.pyi
+++ -
@@ -1,17 +1,25 @@
- from typing import Protocol, overload
+ from typing import Generic, Protocol, overload
+ from typing_extensions import TypeVar
+
+ _T_contra = TypeVar("_T_contra", contravariant=True)
+ _T_co = TypeVar("_T_co", covariant=True)
+ _T = TypeVar("_T", infer_variance=True)
+ _D = TypeVar("_D")
+ _NameT = TypeVar("_NameT", infer_variance=True, bound=str)
+ _QualNameT = TypeVar("_QualNameT", infer_variance=True, bound=str, default=_NameT)

  class Boring: ...

- class CanGetItem[T_contra, T_co](Protocol):
-     def __getitem__(self, k: T_contra, /) -> T_co: ...
+ class CanGetItem(Protocol[_T_contra, _T_co]):
+     def __getitem__(self, k: _T_contra, /) -> _T_co: ...

- class Stack[T]:
-     def push(self, value: T, /) -> None: ...
+ class Stack(Generic[_T, _D]):
+     def push(self, value: _T, /) -> None: ...
      @overload
-     def pop(self, /) -> T: ...
+     def pop(self, /) -> _T: ...
      @overload
-     def pop[D](self, default: D, /) -> T | D: ...
+     def pop(self, default: _D, /) -> _T | _D: ...

- class Named[NameT: str, QualNameT: str = NameT]:
-     __name__: NameT
-     __qualname__: QualNameT
+ class Named(Generic[_NameT, _QualNameT]):
+     __name__: _NameT
+     __qualname__: _QualNameT

Note how TypeVar is (only) imported from typing_extensions here, which wasn't the case in the previous example. This is a consequence of the infer_variance parameter, which has been added in Python 3.12.

Project goals

Here's the alpha version of a prototype of a rough sketch of some initial ideas for the potential goals of unpy:

  1. Towards the past
    • Get frustrated while stubbing scipy
    • Transpile Python 3.13 .pyi stubs to Python 3.10 stubs
    • Package-level analysis and conversion
    • Tooling for stub-only project integration
    • Use this in scipy-stubs
    • Gradually introduce this into numpy
  2. Towards the future
    • Beyond Python: $\text{Unpy} \supset \text{Python}$
    • Language support & tooling for all .py projects
  3. Towards each other
    • Unified typechecking: Fast, reasonable, and language-agnostic

Features

Tooling

  • Target Python versions
    • 3.13
    • 3.12
    • 3.11
    • 3.10
    • 3.9
  • Language support
    • .pyi
    • .py
  • Conversion
    • stdin => stdout
    • module => module
    • package => package
    • project => project (including the pyproject.toml)
  • Configuration
    • --diff: Unified diffs
    • --target: Target Python version, defaults to 3.10
    • Project-based config in pyproject.toml under [tools.unpy]
    • ...
  • Integration
    • File watcher
    • Pre-commit
    • LSP
    • UV
    • VSCode extension
    • (based)mypy plugin
    • Project build tools
    • Configurable type-checker integration
    • Configurable formatter integration, e.g. ruff format
  • Performance
    • Limit conversion to changed files

Stub backporting

  • Python 3.13 => 3.12
    • PEP 742
      • typing.TypeIs => typing_extensions.TypeIs
    • PEP 705
      • typing.ReadOnly => typing_extensions.ReadOnly
    • PEP 702
      • warnings.deprecated => typing_extensions.deprecated
    • PEP 696
      • Backport PEP 695 type signatures with a default
      • typing.NoDefault => typing_extensions.NoDefault
    • Exceptions
      • asyncio.QueueShutDown => builtins.Exception
      • pathlib.UnsupportedOperation => builtins.NotImplementedError
      • queue.ShutDown => builtins.Exception
      • re.PatternError => re.error
    • Typing
      • types.CapsuleType => typing_extensions.CapsuleType
      • typing.{ClassVar,Final} => typing_extensions.{ClassVar,Final} when nested
  • Python 3.12 => 3.11
    • PEP 698
      • typing.override => typing_extensions.override
    • PEP 695
      • Backport type _ aliases
      • Backport generic functions
      • Backport generic classes and protocols
      • typing.TypeAliasType => typing_extensions.TypeAliasType
    • PEP 688
      • collections.abc.Buffer => typing_extensions.Buffer
      • inspect.BufferFlags => int
  • Python 3.11 => 3.10
    • PEP 681
      • typing.dataclass_transform => typing_extensions.dataclass_transform
    • PEP 675
      • typing.LiteralString => typing_extensions.LiteralString
    • PEP 673
      • typing.Self => typing_extensions.Self
    • PEP 655
      • typing.[Not]Required => typing_extensions.[Not]Required
    • PEP 654
      • builtins.BaseExceptionGroup
      • builtins.ExceptionGroup
    • PEP 646
      • typing.TypeVarTuple => typing_extensions.TypeVarTuple
      • typing.Unpack => typing_extensions.Unpack
      • *Ts => typing_extensions.Unpack[Ts] with Ts: TypeVarTuple
    • asyncio
      • asyncio.TaskGroup
    • enum
      • enum.ReprEnum => enum.Enum
      • enum.StrEnum => str & enum.Enum
    • typing
      • typing.Any => typing_extensions.Any if subclassed (not recommended)
  • Generated TypeVars
    • De-duplicate extracted typevar-likes with same name if equivalent
    • Prefix the names of extracted typevar-likes with _
    • Rename incompatible typevar-likes with the same name (#86)

Simplification and refactoring

  • Generic type parameters
    • Convert default=Any with bound=T to default=T
    • Remove bound=Any and bound=object
    • Infer variance of PEP 695 type parameters (#44)
      • If never used, it's redundant (and bivariant) (#46)
      • If constraints are specified, it's invariant
      • If suffixed with _co/_contra, it's covariant/contravariant
      • If used as public instance attribute, it's invariant
      • If only used as return-type (excluding __init__ and __new__), or for read-only attributes, it's covariant
      • If only used as parameter-type, it's contravariant
      • Otherwise, assume it's invariant
  • Methods
    • Default return types for specific "special method" (#55)
    • Transform self method parameters to be positional-only
  • Typing operators
    • type[S] | type[T] => type[S | T]
    • Flatten & de-duplicate unions of literals
    • Remove redundant union values, e.g. bool | int => int

Beyond Python

  • @sealed types (#42)
  • Unified type-ignore comments (#68)
  • Set-based Literal syntax (#76)
  • Reusable method signature definitions (#97, #98)
  • Type-mappings, a DRY alternative to @overload
  • Intersection types (as implemented in basedmypy)
  • Higher-kinded types (see python/typing#548)
  • Inline callable types (inspired by PEP 677)