Total and non-total version of TypedDict

[JanEricNitschke]

Is there any shorter way to creating a total and non-total version of a TypedDict besides just creating two separate ones with the same keys and types?

My first try was to just inherit and have the child as non-total but according to the pep the totality is only supposed to be for the keys defined within that specific dict. Even though the stated use case it not needed anymore with the existence of NotRequired.

So am i right that i have to create two separate ones for this use case?

[erictraut]

So am i right that i have to create two separate ones for this use case?

Yes, you're correct. You need to create two separate ones for this use case. There have been some discussions on the typing forums about providing a way to make this less cumbersome, but those discussions haven't produced any concrete proposals.

[JanEricNitschke]

I have a follow up question on this regarding dict.update() which from looking around seems to be still an open topic on how it should be handled in all cases but here is my scenario anyway.

I have now created two TypedDicts with the same set of keys and want to update the total one by the non-total one and im getting an error (i did not expect).

from typing import TypedDict, final

class Full(TypedDict):
    parse_frames: bool


class NotFull(TypedDict, total=False):

    parse_frames: bool

parser_args: Full = {
            "parse_frames": True,
        }
other_args: NotFull = {
            "parse_frames": False,
}
parser_args.update(other_args)

I understand that this can not necessarily type check because of subclassing. But even then the error i am getting was confusing me a bit.
But i now think this is due to you restricting the arguments of update to be the exact same type to not run into the issue of too many keys?

 error: Argument of type "NotFull" cannot be assigned to parameter "__m" of type "Full" in function "update"
    "parse_frames" is required in "Type[Full]" (reportGeneralTypeIssues)

Even more curious is that mypy seem to type check this without errors.

The pyright error still remains even if i add @final which should make this type safe in my opinion. (mypy still passes it)

Curiously mypy even passes this if i explicitly add more keys to NotFull that are not in Full
https://mypy-play.net/?mypy=latest&python=3.11&gist=3e39b4db2ccb59d341de16b83954d8e9

---

But then again this seems to pass even pyright:

from typing import TypedDict, reveal_type

class Full(TypedDict):
    parse_frames: bool


class NotFull(Full):

    other: bool

parser_args: Full = {
            "parse_frames": True,
        }
other_args: NotFull = {
            "parse_frames": False,
            "other": True,
}
parser_args.update(other_args)

reveal_type(parser_args)  # Type of "parser_args" is "Full"
reveal_type(other_args)  # Type of "other_args" is "NotFull"

[erictraut]

The pyright error still remains even if i add @final

@final should have no effect here. TypedDicts are structural types, like a protocol class. There's no need for a class to derive from a TypedDict to be compatible with it, so @final isn't meaningful in this context.

Pyright doesn't perform any special casing for the update method in TypedDict. The method is defined in typeshed's "typing.pyi" file using a special _TypedDict class. Note that this class doesn't exist at runtime but is purely for the benefit of static type checking. Here's how it defines update:

class _TypedDict(Mapping[str, object], metaclass=ABCMeta):
    ...
    def update(self: _T, __m: _T) -> None: ...

In your code sample above, the expression parser_args.update binds the update method to the object parser_args and replaces the method-local type variable _T with the type Full. The resulting type of the expression parser_args.update is (__m: Full) -> None. That means the argument you pass for parameter __m must be type compatible with Full. The NotFull type is not type compatible, hence the error.

Even more curious is that mypy seem to type check this without errors.

It appears that mypy is performing some special casing for the update method, as evidenced by the inconsistent results I get below:

parser_args.update(other_args) # mypy: No error

x = parser_args.update
reveal_type(x) # mypy: "def (TypedDict('test11.Full', {'parse_frames': builtins.bool}))"
x(other_args) # mypy: Error

Digging into this further, I confirmed that mypy is overriding the signature of update (but doing it in kind of a sketchy manner, which explains the inconsistency above). I'll probably also add special-case handling of update in pyright. The topic came up in a recent discussion about draft PEP 705.

Curiously mypy even passes this if i explicitly add more keys

This is intended. As I mentioned above, TypedDict is a structural type, similar to a protocol. TypedDicts specify the keys/values that must be present, but additional keys (or attributes or methods) can also be present. There is one exception to this rule in the case of direct assignment of literal dictionary expressions, where it was decided that the presence of additional keys should be flagged as an error, since it's likely unintended.

class A(TypedDict):
    a: int

class B(TypedDict):
    a: int
    b: int

a1: A = {"a": 0}
a2: A = {"a": 0, "b": 0} # Error: "b" is an undefined field in type "A"

b1: B = {"a": 0, "b": 0}
a3: A = b1 # No Error

But then again this seems to pass even pyright

That's because pyright is performing assignment-based type narrowing. It knows that other_args has been assigned a value where the parse_frames key has been populated.

[JanEricNitschke]

Super interesting, learning a ton from you, thanks.

So currently with pyright i would have to write my own update method like:

def update_dict(dict1: Full, dict2: NotFull) -> Full:
    for key in dict2:
        dict1[key] = dict2[key]
    return dict1

[erictraut]

So currently with pyright i would have to write my own update...

I've decided to add special-case support for the update method. Pyright now synthesizes a new update method, overriding the method defined in the _TypedDict base class. The parameter type for the synthesized method depends on whether there are any read-only entries defined (see recent discussions about draft PEP 705 if you're interested in the details). If one or more read-only entries are defined in the TypedDict, the update parameter type is Never because updates are not type safe in this situation. If no read-only entries are defined, the parameter type is a "partial" form of the TypedDict. I'm borrowing a concept from TypeScript, which allows any type to be wrapped in Partial<T>, and it marks all of its entries as optional. That's the correct semantics for the update method. There's currently no way to "spell" a partial type in Python, but pyright can internally synthesize this type.

Here's how this looks with the new support:

This support will be included in the next release of pyright.

[JanEricNitschke]

You are absolutely awesome. Will definitely check this out once the release is out.