Overriding the type variable of an inherited custom collection type

[AWhetter]

Let's say I have the following classes:

from collections.abc import Iterator
import pathlib

class Paths:
    _type = pathlib.Path
    def __init__(self) -> None:
        pass

    def __iter__(self) -> Iterator[pathlib.Path]:
        yield self._type(".")

class PosixPaths(Paths):
    _type = pathlib.PosixPath

    def __iter__(self) -> Iterator[pathlib.PosixPath]:
        yield self._type(".")

items = list(Paths())
items = list(PosixPaths())
posix_items = list(PosixPaths())
item: pathlib.PosixPath = posix_items[0]

Is it possible to make Paths inherit from Iterable such that it can represent Iterable[pathlib.Path] and PosixPaths can represent an Iterable[pathlib.PosixPath]? I'm needing to use Sequence instead of Iterable in my implementation, but I'm using Iterable in this example for brevity and I'm assuming that the solution for Iterable can be easily applied to Sequence.

Is it possible for the types to be defined such that PosixPaths does not need to redefine __iter__?

My instinct to solve the above issues is to want to do something like the following:

T = TypeVar("T", bound=pathlib.PurePath)

class _PathsBase(Iterable[T]):
    _type: Type[T]

    def __iter__(self) -> Iterator[T]:
        yield self._type(".")
        yield self._type(".")

class Paths(_PathsBase[pathlib.Path]):
    _type = pathlib.Path

class PosixPaths(Paths[T=pathlib.PosixPath]):  # not real syntax
    _type = pathlib.PosixPath

# or without completely made up syntax

class PosixPaths(Paths, _PathsBase[pathlib.PosixPath]):
    _type = pathlib.PosixPath

item: pathlib.PosixPath = list(PosixPaths())[0]  # Incompatible types in assignment (expression has type "Path", variable has type "PosixPath")

I'm using mypy 1.10.0 (compiled: yes) with Python 3.12.4.

[erictraut]

I think your instincts are leading you in the right direction. The problem is that type variables cannot be used in class variable declarations. That's because all specialized instances of a class share the same class object.

You could do something like this, which stores the type as an instance variable.

Code sample in pyright playground

from collections.abc import Iterator
import pathlib

class PathsBase[T: pathlib.Path]:
    def __init__(self, _type: type[T]) -> None:
        self._type = _type

    def __iter__(self) -> Iterator[T]:
        yield self._type(".")

class Paths(PathsBase[pathlib.Path]):
    def __init__(self) -> None:
        super().__init__(pathlib.Path)

class PosixPaths(PathsBase[pathlib.PosixPath]):
    def __init__(self) -> None:
        super().__init__(pathlib.PosixPath)

items = list(Paths())
items = list(PosixPaths())
posix_items = list(PosixPaths())
item: pathlib.PosixPath = posix_items[0]

[AWhetter]

Thanks for your reply. I didn't make clear in my original question that it's important for PosixPaths to inherit from Paths. Sorry about that. Similarly to how creating a pathlib.Path instance creates either a pathlib.PosixPath or pathlib.WindowsPath depending on which OS it's running on, I'll be making Paths create either a PosixPaths or WindowsPaths. For that to work, I believe PosixPaths must inherit from Paths.

For my own learning, why can't a type variable be used in class variable declarations? I had thought that my use of _type: Type[T] was okay because I hadn't declared it as a class variable, even though I use it like one.

[erictraut]

You can't use a generic type in a class variable because all instances of that class share the same class object — and therefore refer to the same class variable. Consider the following:

class Example[T]:
    x: T

Example[int].x = 1
reveal_type(Example[str].x) # str?
print(Example[str].x)