Bug? TypeDict and covariance [NotABug]

[bukzor]

I don't fully understand functions' co/contravariance rules, so perhaps this is nonsense, but to me, these look like they should be treated identically, since BqRange is a special dict[str, str|None]:

from typing import Mapping
from typing import TypedDict


class BqRange(TypedDict):
    """A JSON-compatible representation of the bigquery RANGE<DATETIME> type. Null values indicate an open

    https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#range_type
    """

    start: str | None
    end: str | None


from typing import Mapping


def bqrange_to_json(x: BqRange) -> Mapping[str, str | None]:
    return x


def bqrange_to_json2(x: dict[str, str | None]) -> Mapping[str, str | None]:
    return x

$ pyright --version
pyright 1.1.379

$ pyright demo.py | sed -r 's#'"$PWD"'/#./#g'
./demo.py
  ./demo.py:21:12 - error: Type "BqRange" is not assignable to return type "Mapping[str, str | None]"
    "BqRange" is not assignable to "Mapping[str, str | None]"
      Type parameter "_VT_co@Mapping" is covariant, but "object" is not a subtype of "str | None"
        Type "object" is not assignable to type "str | None"
          "object" is not assignable to "str"
          "object" is not assignable to "None" (reportReturnType)
1 error, 0 warnings, 0 informations

[erictraut]

Pyright is working correctly here. A TypedDict class is a structural type that defines specific keys and (if they are present) the corresponding value's type. A dictionary can contain additional keys not specified in a TypedDict definition and still be compatible with the TypedDict. That means the value type in the Mapping must be object to account for these possible additional keys. For more details, refer to this portion of the typing spec.

There is a draft PEP 728 that proposes to introduce the concept of a "closed" TypedDict to the Python type system. This PEP hasn't yet been approved by the typing council or the steering council, but pyright implements provisional support for it.

[bukzor]

TIL. Thank you!

My takeaway: TypedDicts are meant to allow additional keys, and so can't do the job of typescript classes.

[bukzor]

I hesistate to report it, since it's doing what I want, but it sounds like this is a bug, in that case?

def bqrange_to_json3(x: BqRange) -> Mapping[str, IsoTimestamp | None]:
    return dict(**x)

[erictraut]

I don't know how the symbol IsoTimestamp is defined in this code sample, so I can't answer your question.

[bukzor]

Apologies! It's just an alias to str, equivalent to my original example.

To reiterate, according to the understanding you've given me, it seems like this should refuse to typecheck (but it does):

from typing import Mapping
from typing import TypedDict


class BqRange(TypedDict):
    """A JSON-compatible representation of the bigquery RANGE<DATETIME> type. Null values indicate an open

    https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#range_type
    """

    start: str | None
    end: str | None

def bqrange_to_json3(x: BqRange) -> Mapping[str, str | None]:
    return dict(**x)

[erictraut]

The typing spec doesn't provide any guidance for the evaluation of a dictionary unpack operator (**) when applied to a TypedDict. This is maybe something that we should eventually include in the spec.

In cases where the spec has nothing to say and the correct behavior is not clear, I ask myself: 1) is pyright's behavior internally consistent?, and 2) what behavior do other type checkers exhibit in this case? Here, I think there's a good argument to be made for changing pyright's current behavior because {**x} should be consistent with dict(**x). Currently they behave differently in pyright. Also, pyright is inconsistent with mypy in the case of dict(**x).

Based on this analysis, I'm going to change pyright so it evaluates the type of dict(**x) as dict[str, object]. That means your code snippet above will generate an error in future versions.

[bukzor]

Sad! But I hope I get a brownie point for being a good citizen :)