From 87fce5f678227af114d6f38096b50ca3b57b7ddb Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Wed, 14 Feb 2024 16:09:56 -0800 Subject: [PATCH 01/80] Added clarifications about illegal uses of `Annotated` special form. (#1618) --- docs/spec/qualifiers.rst | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/spec/qualifiers.rst b/docs/spec/qualifiers.rst index 2d5ad1fdc..07b3a3e74 100644 --- a/docs/spec/qualifiers.rst +++ b/docs/spec/qualifiers.rst @@ -235,6 +235,24 @@ details of the syntax: V == Annotated[list[tuple[int, int]], MaxLen(10)] +* As with most special forms, ``Annotated`` is not type compatible with + ``type`` or ``type[T]``:: + + v1: type[int] = Annotated[int, ""] # Type error + + SmallInt: TypeAlias = Annotated[int, ValueRange(0, 100)] + v2: type[Any] = SmallInt # Type error + +* An attempt to call ``Annotated`` (whether parameterized or not) should be + treated as a type error by type checkers:: + + Annotated() # Type error + Annotated[int, ""](0) # Type error + + SmallInt = Annotated[int, ValueRange(0, 100)] + SmallInt(1) # Type error + + Consuming annotations ^^^^^^^^^^^^^^^^^^^^^ From 62ec871cad97b241a4c5e5cbc483eaaf87f274f6 Mon Sep 17 00:00:00 2001 From: Eric Traut Date: Wed, 14 Feb 2024 17:15:42 -0800 Subject: [PATCH 02/80] Updated tests for spec change and pytype update (#1630) Updated tests for latest version of pytype (2024.02.13). Updated `qualifiers_annotated.py` test to cover recent clarification to the spec. --- .../results/mypy/qualifiers_annotated.toml | 38 +++++++++++------- conformance/results/mypy/version.toml | 2 +- .../results/pyre/qualifiers_annotated.toml | 31 +++++++++------ conformance/results/pyre/version.toml | 2 +- .../results/pyright/qualifiers_annotated.toml | 39 +++++++++++-------- conformance/results/pyright/version.toml | 2 +- .../results/pytype/qualifiers_annotated.toml | 23 ++++++----- conformance/results/pytype/version.toml | 4 +- conformance/results/results.html | 2 +- conformance/src/type_checker.py | 7 +++- conformance/tests/qualifiers_annotated.py | 29 +++++++++++++- 11 files changed, 116 insertions(+), 63 deletions(-) diff --git a/conformance/results/mypy/qualifiers_annotated.toml b/conformance/results/mypy/qualifiers_annotated.toml index 44adc1ec8..87f8d278f 100644 --- a/conformance/results/mypy/qualifiers_annotated.toml +++ b/conformance/results/mypy/qualifiers_annotated.toml @@ -3,24 +3,32 @@ notes = """ Does not allow ClassVar to be nested within Annotated. Does not allow Final to be nested within Annotated. Does not allow Required and NotRequired to be nested within Annotated. +Does not reject type[T] compatibility for type alias defined with Annotated. +Does not reject call of type alias defined with Annotated. """ output = """ -qualifiers_annotated.py:41: error: Bracketed expression "[...]" is not valid as a type [valid-type] -qualifiers_annotated.py:42: error: Syntax error in type annotation [syntax] -qualifiers_annotated.py:42: note: Suggestion: Is there a spurious trailing comma? -qualifiers_annotated.py:43: error: Invalid type comment or annotation [valid-type] -qualifiers_annotated.py:44: error: Invalid type comment or annotation [valid-type] +qualifiers_annotated.py:43: error: Bracketed expression "[...]" is not valid as a type [valid-type] +qualifiers_annotated.py:44: error: Syntax error in type annotation [syntax] +qualifiers_annotated.py:44: note: Suggestion: Is there a spurious trailing comma? qualifiers_annotated.py:45: error: Invalid type comment or annotation [valid-type] qualifiers_annotated.py:46: error: Invalid type comment or annotation [valid-type] qualifiers_annotated.py:47: error: Invalid type comment or annotation [valid-type] -qualifiers_annotated.py:48: error: Name "var1" is not defined [name-defined] -qualifiers_annotated.py:49: error: Invalid type: try using Literal[True] instead? [valid-type] -qualifiers_annotated.py:50: error: Invalid type: try using Literal[1] instead? [valid-type] -qualifiers_annotated.py:51: error: Invalid type comment or annotation [valid-type] -qualifiers_annotated.py:52: error: Invalid type comment or annotation [valid-type] -qualifiers_annotated.py:62: error: Annotated[...] must have exactly one type argument and at least one annotation [valid-type] -qualifiers_annotated.py:73: error: Invalid type: ClassVar nested inside other type [valid-type] -qualifiers_annotated.py:75: error: Final can be only used as an outermost qualifier in a variable annotation [valid-type] -qualifiers_annotated.py:85: error: Required[] can be only used in a TypedDict definition [valid-type] -qualifiers_annotated.py:87: error: NotRequired[] can be only used in a TypedDict definition [valid-type] +qualifiers_annotated.py:48: error: Invalid type comment or annotation [valid-type] +qualifiers_annotated.py:49: error: Invalid type comment or annotation [valid-type] +qualifiers_annotated.py:50: error: Name "var1" is not defined [name-defined] +qualifiers_annotated.py:51: error: Invalid type: try using Literal[True] instead? [valid-type] +qualifiers_annotated.py:52: error: Invalid type: try using Literal[1] instead? [valid-type] +qualifiers_annotated.py:53: error: Invalid type comment or annotation [valid-type] +qualifiers_annotated.py:54: error: Invalid type comment or annotation [valid-type] +qualifiers_annotated.py:64: error: Annotated[...] must have exactly one type argument and at least one annotation [valid-type] +qualifiers_annotated.py:76: error: Incompatible types in assignment (expression has type "object", variable has type "type[Any]") [assignment] +qualifiers_annotated.py:84: error: Argument 1 to "func4" has incompatible type "object"; expected "type[Never]" [arg-type] +qualifiers_annotated.py:91: error: "" not callable [operator] +qualifiers_annotated.py:92: error: "object" not callable [operator] +qualifiers_annotated.py:98: error: Invalid type: ClassVar nested inside other type [valid-type] +qualifiers_annotated.py:100: error: Final can be only used as an outermost qualifier in a variable annotation [valid-type] +qualifiers_annotated.py:110: error: Required[] can be only used in a TypedDict definition [valid-type] +qualifiers_annotated.py:112: error: NotRequired[] can be only used in a TypedDict definition [valid-type] +qualifiers_annotated.py:119: error: Cannot redefine "T" as a type variable [misc] +qualifiers_annotated.py:119: error: Invalid assignment target [misc] """ diff --git a/conformance/results/mypy/version.toml b/conformance/results/mypy/version.toml index fae14bea2..cde8c6baa 100644 --- a/conformance/results/mypy/version.toml +++ b/conformance/results/mypy/version.toml @@ -1,2 +1,2 @@ version = "mypy 1.8.0" -test_duration = 2.0 +test_duration = 1.4 diff --git a/conformance/results/pyre/qualifiers_annotated.toml b/conformance/results/pyre/qualifiers_annotated.toml index e0d4e1149..b4ddd0ee5 100644 --- a/conformance/results/pyre/qualifiers_annotated.toml +++ b/conformance/results/pyre/qualifiers_annotated.toml @@ -1,18 +1,25 @@ conformant = "Partial" notes = """ Does not reject Annotated with a single parameter. +Does not reject call of Annotated with no type arguments. """ output = """ -qualifiers_annotated.py:41:6 Undefined or invalid type [11]: Annotation `` is not defined as a type. -qualifiers_annotated.py:42:6 Invalid type [31]: Expression `typing.Annotated[(((int, str)), "")]` is not a valid type. -qualifiers_annotated.py:43:6 Invalid type [31]: Expression `typing.Annotated[(comprehension(int for generators(generator($target$i in range(1) if ))), "")]` is not a valid type. -qualifiers_annotated.py:44:6 Invalid type [31]: Expression `typing.Annotated[({ "a":"b" }, "")]` is not a valid type. -qualifiers_annotated.py:45:6 Invalid type [31]: Expression `typing.Annotated[(lambda () (int)(), "")]` is not a valid type. -qualifiers_annotated.py:46:6 Invalid type [31]: Expression `typing.Annotated[([int][0], "")]` is not a valid type. -qualifiers_annotated.py:47:6 Invalid type [31]: Expression `typing.Annotated[(int if 1 < 3 else str, "")]` is not a valid type. -qualifiers_annotated.py:48:16 Unbound name [10]: Name `var1` is used but not defined in the current scope. -qualifiers_annotated.py:49:6 Invalid type [31]: Expression `typing.Annotated[(True, "")]` is not a valid type. -qualifiers_annotated.py:50:7 Invalid type [31]: Expression `typing.Annotated[(1, "")]` is not a valid type. -qualifiers_annotated.py:51:7 Invalid type [31]: Expression `typing.Annotated[(list or set, "")]` is not a valid type. -qualifiers_annotated.py:52:7 Invalid type [31]: Expression `typing.Annotated[(f"{"int"}", "")]` is not a valid type. +qualifiers_annotated.py:43:6 Undefined or invalid type [11]: Annotation `` is not defined as a type. +qualifiers_annotated.py:44:6 Invalid type [31]: Expression `typing.Annotated[(((int, str)), "")]` is not a valid type. +qualifiers_annotated.py:45:6 Invalid type [31]: Expression `typing.Annotated[(comprehension(int for generators(generator($target$i in range(1) if ))), "")]` is not a valid type. +qualifiers_annotated.py:46:6 Invalid type [31]: Expression `typing.Annotated[({ "a":"b" }, "")]` is not a valid type. +qualifiers_annotated.py:47:6 Invalid type [31]: Expression `typing.Annotated[(lambda () (int)(), "")]` is not a valid type. +qualifiers_annotated.py:48:6 Invalid type [31]: Expression `typing.Annotated[([int][0], "")]` is not a valid type. +qualifiers_annotated.py:49:6 Invalid type [31]: Expression `typing.Annotated[(int if 1 < 3 else str, "")]` is not a valid type. +qualifiers_annotated.py:50:16 Unbound name [10]: Name `var1` is used but not defined in the current scope. +qualifiers_annotated.py:51:6 Invalid type [31]: Expression `typing.Annotated[(True, "")]` is not a valid type. +qualifiers_annotated.py:52:7 Invalid type [31]: Expression `typing.Annotated[(1, "")]` is not a valid type. +qualifiers_annotated.py:53:7 Invalid type [31]: Expression `typing.Annotated[(list or set, "")]` is not a valid type. +qualifiers_annotated.py:54:7 Invalid type [31]: Expression `typing.Annotated[(f"{"int"}", "")]` is not a valid type. +qualifiers_annotated.py:76:33 Incompatible parameter type [6]: In call `typing.GenericMeta.__getitem__`, for 1st positional argument, expected `Tuple[]` but got `Tuple[Type[int], str]`. Expected has length 0, but actual has length 2. +qualifiers_annotated.py:77:0 Incompatible variable type [9]: not_type2 is declared to have type `Type[typing.Any]` but is used as type `TypeAlias`. +qualifiers_annotated.py:84:16 Incompatible parameter type [6]: In call `typing.GenericMeta.__getitem__`, for 1st positional argument, expected `Tuple[]` but got `Tuple[Type[str], str]`. Expected has length 0, but actual has length 2. +qualifiers_annotated.py:85:6 Incompatible parameter type [6]: In call `func4`, for 1st positional argument, expected `Type[Variable[T]]` but got `TypeAlias`. +qualifiers_annotated.py:92:10 Incompatible parameter type [6]: In call `typing.GenericMeta.__getitem__`, for 1st positional argument, expected `Tuple[]` but got `Tuple[Type[int], str]`. Expected has length 0, but actual has length 2. +qualifiers_annotated.py:93:0 Call error [29]: `TypeAlias` is not a function. """ diff --git a/conformance/results/pyre/version.toml b/conformance/results/pyre/version.toml index 8d7091d95..e4f3bd0c9 100644 --- a/conformance/results/pyre/version.toml +++ b/conformance/results/pyre/version.toml @@ -1,2 +1,2 @@ version = "pyre 0.9.19" -test_duration = 3.3 +test_duration = 2.8 diff --git a/conformance/results/pyright/qualifiers_annotated.toml b/conformance/results/pyright/qualifiers_annotated.toml index 8eb361f30..cef4b886a 100644 --- a/conformance/results/pyright/qualifiers_annotated.toml +++ b/conformance/results/pyright/qualifiers_annotated.toml @@ -1,22 +1,29 @@ conformant = "Pass" output = """ -qualifiers_annotated.py:41:17 - error: List expression not allowed for this type argument -qualifiers_annotated.py:42:17 - error: Expected type expression but received "tuple[tuple[type[int], type[str]]]" (reportGeneralTypeIssues) qualifiers_annotated.py:43:17 - error: List expression not allowed for this type argument -qualifiers_annotated.py:43:18 - error: Expected type expression but received "Generator[type[int], None, None]" (reportGeneralTypeIssues) -qualifiers_annotated.py:44:17 - error: Expected type expression but received "dict[str, str]" (reportGeneralTypeIssues) -qualifiers_annotated.py:44:17 - error: Dictionary expression not allowed in type annotation -qualifiers_annotated.py:45:17 - error: Call expression not allowed in type expression (reportInvalidTypeForm) -qualifiers_annotated.py:46:17 - error: List expression not allowed in type annotation +qualifiers_annotated.py:44:17 - error: Expected type expression but received "tuple[tuple[type[int], type[str]]]" (reportGeneralTypeIssues) +qualifiers_annotated.py:45:17 - error: List expression not allowed for this type argument +qualifiers_annotated.py:45:18 - error: Expected type expression but received "Generator[type[int], None, None]" (reportGeneralTypeIssues) +qualifiers_annotated.py:46:17 - error: Expected type expression but received "dict[str, str]" (reportGeneralTypeIssues) +qualifiers_annotated.py:46:17 - error: Dictionary expression not allowed in type annotation +qualifiers_annotated.py:47:17 - error: Call expression not allowed in type expression (reportInvalidTypeForm) +qualifiers_annotated.py:48:17 - error: List expression not allowed in type annotation   Use List[T] to indicate a list type or Union[T1, T2] to indicate a union type (reportInvalidTypeForm) -qualifiers_annotated.py:46:17 - error: Expected type expression but received "list[type[int]]" (reportGeneralTypeIssues) -qualifiers_annotated.py:47:17 - error: Ternary expression not allowed in type annotation (reportInvalidTypeForm) -qualifiers_annotated.py:48:17 - error: "var1" is not defined (reportUndefinedVariable) -qualifiers_annotated.py:49:17 - error: Expected type expression but received "Literal[True]" (reportGeneralTypeIssues) -qualifiers_annotated.py:50:18 - error: Expected type expression but received "Literal[1]" (reportGeneralTypeIssues) -qualifiers_annotated.py:51:18 - error: Binary operator not allowed in type annotation (reportInvalidTypeForm) -qualifiers_annotated.py:52:18 - error: Expected expression -qualifiers_annotated.py:52:18 - error: Tuple expression not allowed in type annotation +qualifiers_annotated.py:48:17 - error: Expected type expression but received "list[type[int]]" (reportGeneralTypeIssues) +qualifiers_annotated.py:49:17 - error: Ternary expression not allowed in type annotation (reportInvalidTypeForm) +qualifiers_annotated.py:50:17 - error: "var1" is not defined (reportUndefinedVariable) +qualifiers_annotated.py:51:17 - error: Expected type expression but received "Literal[True]" (reportGeneralTypeIssues) +qualifiers_annotated.py:52:18 - error: Expected type expression but received "Literal[1]" (reportGeneralTypeIssues) +qualifiers_annotated.py:53:18 - error: Binary operator not allowed in type annotation (reportInvalidTypeForm) +qualifiers_annotated.py:54:18 - error: Expected expression +qualifiers_annotated.py:54:18 - error: Tuple expression not allowed in type annotation   Use tuple[T1, ..., Tn] to indicate a tuple type or Union[T1, T2] to indicate a union type (reportInvalidTypeForm) -qualifiers_annotated.py:62:8 - error: Expected one type argument and one or more annotations for "Annotated" +qualifiers_annotated.py:64:8 - error: Expected one type argument and one or more annotations for "Annotated" +qualifiers_annotated.py:76:24 - error: Expression of type "type[int]" cannot be assigned to declared type "type[Any]" (reportAssignmentType) +qualifiers_annotated.py:77:24 - error: Expression of type "SmallInt" cannot be assigned to declared type "type[Any]" (reportAssignmentType) +qualifiers_annotated.py:84:7 - error: Argument of type "type[str]" cannot be assigned to parameter "x" of type "type[T@func4]" in function "func4" (reportArgumentType) +qualifiers_annotated.py:85:7 - error: Argument of type "SmallInt" cannot be assigned to parameter "x" of type "type[T@func4]" in function "func4" (reportArgumentType) +qualifiers_annotated.py:91:1 - error: "Annotated" cannot be instantiated (reportCallIssue) +qualifiers_annotated.py:92:1 - error: Object of type "type[Annotated]" is not callable (reportCallIssue) +qualifiers_annotated.py:93:1 - error: Object of type "type[Annotated]" is not callable (reportCallIssue) """ diff --git a/conformance/results/pyright/version.toml b/conformance/results/pyright/version.toml index 26233019e..b316b378d 100644 --- a/conformance/results/pyright/version.toml +++ b/conformance/results/pyright/version.toml @@ -1,2 +1,2 @@ version = "pyright 1.1.350" -test_duration = 1.5 +test_duration = 1.4 diff --git a/conformance/results/pytype/qualifiers_annotated.toml b/conformance/results/pytype/qualifiers_annotated.toml index d73893788..81ecee57c 100644 --- a/conformance/results/pytype/qualifiers_annotated.toml +++ b/conformance/results/pytype/qualifiers_annotated.toml @@ -1,17 +1,20 @@ conformant = "Partial" notes = """ Does not reject some illegal type expression forms used in Annotated. +Does not report type incompatibility between Annotated and type[T]. +Does not reject call of Annotated. Does not allow TypeVar to be used in type alias when wrapped with Annotated. """ output = """ -File "qualifiers_annotated.py", line 41, in : Invalid type annotation '[int, str]' for Bad1 [invalid-annotation] -File "qualifiers_annotated.py", line 42, in : Invalid type annotation '((int, str),)' for Bad2 [invalid-annotation] -File "qualifiers_annotated.py", line 43, in : Invalid type annotation '' for Bad3 [invalid-annotation] -File "qualifiers_annotated.py", line 44, in : Invalid type annotation "{'a': 'b'}" for Bad4 [invalid-annotation] -File "qualifiers_annotated.py", line 48, in : Name 'var1' is not defined [name-error] -File "qualifiers_annotated.py", line 49, in : Invalid type annotation 'True' for Bad9 [invalid-annotation] -File "qualifiers_annotated.py", line 50, in : Invalid type annotation '1' for Bad10 [invalid-annotation] -File "qualifiers_annotated.py", line 52, in : Invalid type annotation '' for Bad12 [invalid-annotation] -File "qualifiers_annotated.py", line 62, in : Invalid type annotation 'Annotated' [invalid-annotation] -File "qualifiers_annotated.py", line 96, in : Invalid TypeVar: TypeVar('T') must be stored as 'T', not 'TA3' [invalid-typevar] +File "qualifiers_annotated.py", line 43, in : Invalid type annotation '[int, str]' for Bad1 [invalid-annotation] +File "qualifiers_annotated.py", line 44, in : Invalid type annotation '((int, str),)' for Bad2 [invalid-annotation] +File "qualifiers_annotated.py", line 45, in : Invalid type annotation '' for Bad3 [invalid-annotation] +File "qualifiers_annotated.py", line 46, in : Invalid type annotation "{'a': 'b'}" for Bad4 [invalid-annotation] +File "qualifiers_annotated.py", line 50, in : Name 'var1' is not defined [name-error] +File "qualifiers_annotated.py", line 51, in : Invalid type annotation 'True' for Bad9 [invalid-annotation] +File "qualifiers_annotated.py", line 52, in : Invalid type annotation '1' for Bad10 [invalid-annotation] +File "qualifiers_annotated.py", line 54, in : Invalid type annotation '' for Bad12 [invalid-annotation] +File "qualifiers_annotated.py", line 64, in : Invalid type annotation 'Annotated' [invalid-annotation] +File "qualifiers_annotated.py", line 91, in : 'Annotated' object is not callable [not-callable] +File "qualifiers_annotated.py", line 121, in : Invalid TypeVar: TypeVar('T') must be stored as 'T', not 'TA3' [invalid-typevar] """ diff --git a/conformance/results/pytype/version.toml b/conformance/results/pytype/version.toml index a22abc607..ad2b34e9b 100644 --- a/conformance/results/pytype/version.toml +++ b/conformance/results/pytype/version.toml @@ -1,2 +1,2 @@ -version = "pytype 2024.02.09" -test_duration = 30.3 +version = "pytype 2024.02.13" +test_duration = 31.6 diff --git a/conformance/results/results.html b/conformance/results/results.html index b3de96bb9..bb7beddca 100644 --- a/conformance/results/results.html +++ b/conformance/results/results.html @@ -156,7 +156,7 @@

Python Type System Conformance Test Results

-
 
mypy 1.8.0
2.0sec
pyright 1.1.350
1.5sec
pyre 0.9.19
3.3sec
pytype 2024.02.09
30.3sec
Type annotations
     annotations_coroutinesPassPass
Partial

Does not evaluate correct type for async function.

Partial

Does not evaluate correct type for async function.

     annotations_forward_refs
Partial

Does not report error for a forward reference that is not enclosed in quotes.

Does not report error for use of quoted type with "|" operator (runtime error).

Incorrectly generates error for quoted type defined in class scope.

Does not treat triple-quoted forward reference annotation as implicitly parenthesized.

Pass
Partial

Does not report error for a forward reference that is not enclosed in quotes.

Does not report error for use of quoted type with "|" operator (runtime error).

Does not reject f-string in quoted type annotation.

Incorrectly generates error for quoted type defined in class scope.

Does not generate error for unquoted type defined in class scope.

Does not treat triple-quoted forward reference annotation as implicitly parenthesized.

Partial

Does not reject some illegal type expression forms when quoted.

Incorrectly generates error for quoted type defined in class scope.

Evaluates incorrect type for class variable annotated with quoted type expression.

Does not treat triple-quoted forward reference annotation as implicitly parenthesized.

     annotations_generators
Partial

Does not report incompatible Generator type in `yield from` statement.

Pass
Partial

Does not report invalid return type for generator when function implicitly returns None.

Incorrectly evaluates type of call to async generator.

Partial

Does not report invalid return type for generator when function implicitly returns None.

Reports invalid error when return type of generator is annotated as a compatible protocol.

Does not report type violation in `yield from` statement.

     annotations_methods
Pass*

Type evaluation differs from other type checkers because of ambiguity in the spec related to method bindings.

Pass*

Type evaluation differs from other type checkers because of ambiguity in the spec related to method bindings.

Pass*

Type evaluation differs from other type checkers because of ambiguity in the spec related to method bindings.

Pass*

Type evaluation differs from other type checkers because of ambiguity in the spec related to method bindings.

     annotations_typeexprPassPassPass
Partial

Does not reject call expressions in type annotation.

Does not reject call lambda expression in type annotation.

Does not reject list expression in type annotation.

Does not reject ternary expression in type annotation.

Does not reject f-string in type annotation.

Does not reject module in type annotation.

Special types in annotations
     specialtypes_anyPassPass
Partial

Does not treat missing type argument as Any in generic type.

Does not support Any as a base class.

Pass
     specialtypes_neverPassPass
Partial

Does not treat Never as compatible with all other types.

Unsupported

Does not understand NoReturn or Never.

     specialtypes_nonePassPass
Partial

Does not correctly handle type annotation type[None].

Partial

Does not detect type incompatibility between None and type[None].

Does not detect type incompatibility between None and incompatible protocol.

     specialtypes_promotionsPassPass
Partial

Does not reject use of attribute that is compatible only with float.

Pass
     specialtypes_type
Partial

Does not treat `type` same as `type[Any]` for assert_type.

Does not allow access to unknown attributes from object of type `type[Any]`.

Pass
Partial

Does not reject Callable when passed to type[T].

Does not treat `type` same as `type[Any]` for assert_type.

Does not allow access to unknown attributes from object of type `type[Any]`.

Does not reject access to unknown attributes from object of type `Type[object]`.

Reports type incompatibility between `type` and `Callable[..., Any]`.

Partial

Does not reject Callable when passed to type[T].

Does not allow access to known attributes from object of type `type[Any]`.

Generics
     generics_base_classPassPass
Pass*

Doesn't allow using generic in assert_type expression.

Partial

False negative on passing SymbolTable to dict[str, list[object]].

     generics_basicPassPass
Partial

False positives in examples using constrained type variables.

False negative in custom map example.

False positive using `iter`.

False negative for generic metaclass.

Partial

False positives in examples using constrained type variables.

False negative for generic metaclass.

     generics_paramspec_basic
Partial

Does not reject ParamSpec when used "bare" in type alias definition.

Pass
Partial

Does not enforce name consistency for ParamSpec assigned to identifier.

Incorrectly reports error for legitimate use of ParamSpec in generic type alias.

Does not reject ParamSpec when used in various invalid locations.

Unsupported

Does not support ParamSpec.

     generics_paramspec_components
Partial

Does not report illegal use of "P.args" on normal parameter.

Does not report error when P.args is specified but P.kwargs is missing.

Does not report error when P is out of scope and P.args and P.kwargs is used.

Does not report error when keyword argument is specified between P.args and P.kwargs.

Does not report error when calling callable and argument is missing for concatenated parameters.

Pass
Partial

Does not report illegal use of "P.args" on normal parameter.

Does not report error when P.args is specified but P.kwargs is missing.

Does not report error when P is out of scope and P.args and P.kwargs is used.

Does not report error when calling callback defined with ParamSpec with incorrect arguments.

Does not report error when keyword argument is specified between P.args and P.kwargs.

Does not report error when calling callable and argument is missing for concatenated parameters.

Unsupported

Does not support ParamSpec.

     generics_paramspec_semanticsPass
Pass*

Constraint solver doesn't find common type for two signatures captured by a single ParamSpec (allowed).

Partial

Constraint solver doesn't find common type for two signatures captured by a single ParamSpec (allowed).

Reports error for legitimate Callable type annotation that uses Concatenate.

Does not evaluate the correct type for call of Callable defined with Concatenate.

Unsupported

Does not support ParamSpec.

     generics_paramspec_specializationPassPass
Partial

Reports error for legitimate use of ParamSpec and Concatenate in function signature.

Reports error for legitimate specialization of generic class parameterized with ParamSpec.

Partial

Rejects valid specialization of ParamSpec using list expression.

Does not reject invalid specialization of class with both TypeVar and ParamSpec.

Reports error for valid method call involving ParamSpec.

     generics_scoping
Partial

False negative on generic class nested within generic class with same type variable.

Pass
Partial

False negative on generic class nested within generic function with same type variable.

False negative on generic class nested within generic class with same type variable.

Pass
     generics_self_advanced
Partial

Does not infer the type of an unannotated `self` parameter to be type `Self`.

Does not retain `Self` when calling method that returns `Self`.

Does not infer the type of an unannotated `cls` parameter to be type `type[Self]`.

Does not retain `Self` when accessing attribute through `type[Self]`.

Pass
Unsupported

Does not understand `Self` type.

Unsupported

Does not understand `Self` type.

     generics_self_attributesPassPass
Unsupported

Does not understand `Self` type.

Unsupported

Does not understand `Self` type.

     generics_self_basic
Partial

Does not properly handle constructor call through `cls` parameter.

Pass
Unsupported

Does not understand `Self` type.

Unsupported

Does not understand `Self` type.

     generics_self_protocolsPassPass
Partial

Does not reject protocol compatibility due to method `Self` return type.

Partial

Does not reject protocol compatibility due to method `Self` return type.

     generics_self_usagePassPass
Unsupported

Does not understand `Self` type.

Unsupported

Does not understand `Self` type.

     generics_syntax_compatibility
Unsupported

Type parameter syntax not yet support.

Pass
Unsupported

Type parameter syntax not yet support.

Unsupported

Type parameter syntax not yet support.

     generics_syntax_declarations
Unsupported

Type parameter syntax not yet support.

Pass
Unsupported

Type parameter syntax not yet support.

Unsupported

Type parameter syntax not yet support.

     generics_syntax_infer_variance
Unsupported

Type parameter syntax not yet support.

Pass
Unsupported

Type parameter syntax not yet support.

Unsupported

Type parameter syntax not yet support.

     generics_syntax_scoping
Unsupported

Type parameter syntax not yet support.

Pass
Unsupported

Type parameter syntax not yet support.

Unsupported

Type parameter syntax not yet support.

     generics_type_erasure
Partial

Infers Node[Never] instead of Node[Any] when argument is not provided.

False negative on instance attribute access on type(node).

Pass
Partial

Doesn't allow using Node[Any] in assert_type expression.

False negatives on instance attribute access on the type.

Unsupported
     generics_typevartuple_args
Partial

Does not enforce that tuples captured by TypeVarTuple are same type.

Pass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_basic
Partial

Does not enforce that tuples captured by TypeVarTuple are same length.

Does not enforce that tuples captured by TypeVarTuple are same type.

Does not enforce that tuples captured by TypeVarTuple are invariant in non-tuple class.

Pass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_callablePassPass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_concatPassPass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_overloadsPassPass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_specialization
Partial

Incorrectly specializes generic alias that includes a TypeVar and TypeVarTuple if no type arguments are provided.

Rejects specialization of generic type alias defined as a tuple containing a TypeVar.

"More than one Unpack" error message has no line number.

Pass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_unpack
Partial

Does not reject multiple unpack operators in a tuple.

Pass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_variance
Partial

Does not reject use of class-scoped TypeVar used in a base class when variance is incompatible.

Pass
Partial

Does not reject a TypeVar that is defined as both covariant and contravariant.

Does not reject use of class-scoped TypeVar used in a base class when variance is incompatible.

Unsupported

Does not support covariant or contravariant TypeVars.

     generics_variance_inference
Unsupported

Type parameter syntax not yet support.

Pass
Unsupported

Type parameter syntax not yet support.

Unsupported

Type parameter syntax not yet support.

Type qualifiers
     qualifiers_annotated
Partial

Does not allow ClassVar to be nested within Annotated.

Does not allow Final to be nested within Annotated.

Does not allow Required and NotRequired to be nested within Annotated.

Pass
Partial

Does not reject Annotated with a single parameter.

Partial

Does not reject some illegal type expression forms used in Annotated.

Does not allow TypeVar to be used in type alias when wrapped with Annotated.

     qualifiers_final_annotation
Partial

Does not treat use of Final name as if it was replaced by the literal in NamedTuple definition.

Does not allow conditional assignment of Final instance variable in __init__ method.

Does not allow redefinition of private class variable that is marked Final in parent class.

Does not report modification of local Final variable via "for" statement.

Pass
Partial

Does not report Final variable with missing initialization in module scope.

Does not report error for invalid nesting of Final and ClassVar.

Does not treat use of Final name as if it was replaced by the literal in NamedTuple definition.

Partial

Does not report Final variable with missing initialization.

Does not reject Final instance variable declared outside of __init__ method.

Does not reject modification of global variable declared Final.

Does not reject modification of local variable declared Final.

     qualifiers_final_decoratorPassPass
Partial

Reports error for overloaded method implementation marked @final if its overloads do not.

Does not report error for overloaded @final method defined in stub file.

Reports misleading error when overload is marked @final but implementation is not.

Partial

Does not report error for overloaded @final method defined in stub file.

Does not report error for overload that is marked @final when implementation is not.

Class type compatibility
     classes_classvar
Partial

Internal error if TypeVarTuple is used in ClassVar.

Does not reject use of ParamSpec in ClassVar.

Rejects ClassVar nested in Annotated.

Does not reject use of ClassVar in TypeAlias definition.

Does not infer type of ClassVar from assignment if no type is provided.

Pass
Partial

Does not reject use of TypeVar in ClassVar.

Does not reject use of ParamSpec in ClassVar.

Does not reject use of ClassVar as a generic type argument.

Does not reject use of ClassVar in parameter type annotation.

Does not reject use of ClassVar in local variable annotation.

Does not reject use of ClassVar in instance variable annotation.

Does not reject use of ClassVar in return type annotation.

Does not reject use of ClassVar in type alias definition.

Partial

Does not reject use of TypeVar in ClassVar.

Does not reject use of ParamSpec in ClassVar.

Does not reject use of ClassVar as a generic type argument.

Rejects initialization of ClassVar if no type argument is provided.

Does not reject use of ClassVar in parameter type annotation.

Does not reject use of ClassVar in local variable annotation.

Does not reject use of ClassVar in instance variable annotation.

Does not reject use of ClassVar in return type annotation.

Does not reject use of ClassVar in type alias definition.

Does not reject assignment of ClassVar through instance of class.

     classes_override
Partial

Does not handle case where parent class derives from Any.

Pass
Unsupported

Does not yet support the @override decorator.

Unsupported

Does not yet support the @override decorator.

Type aliases
     aliases_explicit
Partial

Does not reject specialization of type alias that has already been implicitly specialized.

Pass
Partial

Incorrectly reports error for type alias defined with ParamSpec.

Incorrectly rejects some valid type aliases when used in annotations.

Incorrectly evaluates generic type alias with ParamSpec with missing type argument.

Does not report some illegal annotation forms as invalid type aliases.

Does not report invalid specialization of generic type aliases.

Incorrectly rejects import alias of `TypeAlias` when used to define type alias.

Does not report invalid specialization of already-specialized generic type alias.

Partial

Incorrectly reports error for type alias defined with ParamSpec.

Does not report invalid specialization of generic type alias with bound TypeVar.

Incorrectly evaluates generic type alias with ParamSpec with missing type argument.

Does not report some illegal annotation forms as invalid type aliases.

Does not report invalid specialization of already-specialized generic type alias.

     aliases_implicitPassPass
Partial

Incorrectly reports error for type alias defined with ParamSpec.

Incorrectly rejects some valid type aliases when used in annotations.

Incorrectly evaluates generic type alias with ParamSpec with missing type argument.

Does not report invalid specialization of generic type aliases.

Does not report error for attempt to instantiate union type alias.

Does not report invalid specialization of already-specialized generic type alias.

Partial

Incorrectly reports error for type alias defined with ParamSpec.

Does not report invalid specialization of generic type alias with bound TypeVar.

Incorrectly evaluates generic type alias with ParamSpec with missing type argument.

Allows some illegal annotation forms to be interpreted as valid type aliases.

Does not report invalid specialization of already-specialized generic type alias.

     aliases_newtypePassPass
Partial

Does not reject use of NewType in `isinstance` call.

Does not reject use of NewType in class definition statement.

Does not report inconsistency between name of NewType and assigned identifier name.

Does not reject use of NewType with generic class with TypeVar.

Does not reject use of NewType with protocol class.

Does not reject use of NewType with TypedDict class.

Does not reject use of NewType with Any.

Partial

Does not reject use of NewType in `isinstance` call.

Does not reject use of NewType in class definition statement.

Does not report inconsistency between name of NewType and assigned identifier name.

Does not reject use of NewType with generic class with TypeVar.

Does not reject use of NewType with protocol class.

Does not reject use of NewType with TypedDict class.

Does not reject use of NewType with Any.

     aliases_recursivePassPass
Partial

Does not properly handle some recursive type aliases.

Does not properly handle specialization of generic recursive type aliases.

Partial

Does not detect type violation for some deeply-nested types.

Does not properly handle `|` for unions in some recursive type alias definitions.

Does not detect cyclical references in recursive type alias definition.

     aliases_type_statement
Unsupported

Does not support `type` statement.

Pass
Unsupported

Does not support `type` statement.

Unsupported

Does not support `type` statement.

     aliases_typealiastype
Unsupported

Support for TypeAliasType is not implemented.

Pass
Unsupported

Support for TypeAliasType is not implemented.

Unsupported

Support for TypeAliasType is not implemented.

     aliases_variancePassPassPass
Unsupported

Does not detect variance incompatibility.

Literals
     literals_interactions
Partial

Does not narrow type of `x` with `x in Literal` type guard pattern.

Pass
Partial

Does not detect out-of-bound tuple literal index.

Does not narrow type of `x` with `x in Literal` type guard pattern.

Does not narrow type of `x` with `x == Literal` type guard pattern.

Partial

Incorrectly rejects some legal Literal annotations.

Does not reject some illegal Literal annotations.

Does not use Literal to distinguish overloads.

Does not narrow based on `x is Literal` type guard pattern.

Does not narrow based on `x == Literal` type guard pattern.

     literals_literalstring
Unsupported

Support for `LiteralString` is not implemented.

PassPass
Unsupported

Does not understand `LiteralString` special form.

     literals_parameterizations
Partial

Rejects integer Literal with unary `+` operator.

Does not reject tuple within Literal.

Pass
Partial

Does not support type aliases in Literal type expression.

Does not support nested Literal type expression.

Does not reject unary + operator in Literal type expression.

Does not reject tuple in Literal type expression.

Does not reject "bare" Literal in type expression.

Unsupported

Does not understand `Literal` type annotation.

     literals_semanticsPassPass
Partial

Does not reject augmented operation that modifies literal value.

Unsupported

Does not understand `Literal` type annotation.

Protocols
     protocols_class_objectsPassPass
Partial

Does not reject protocol class assigned to type[Proto].

Incorrectly reports some class objects as incompatible with a protocol.

Fails to report some class objects as incompatible with a protocol.

Partial

Does not reject protocol class assigned to type[Proto].

Incorrectly reports some class objects as incompatible with a protocol.

Fails to report some class objects as incompatible with a protocol.

     protocols_definition
Partial

Does not detect protocol mismatch if concrete method is missing annotations.

Does not detect protocol mismatch if concrete method's parameters are position-only.

Pass
Partial

Does not reject ClassVar in concrete class when attribute in protocol is not ClassVar or vice versa.

Does not reject read-only property in concrete class when attribute in protocol is mutable.

Does not reject covariant attribute type when protocol attribute is mutable.

Does not reject read-only property in concrete class when protocol has settable property.

Does not reject immutable named tuple attribute in concrete class when protocol attribute is mutable.

Does not reject immutable frozen dataclass attribute in concrete class when protocol attribute is mutable.

Partial

Reports errors for protocol static method with "..." implementation.

Does not report error when instance variable is set through "self" access in protocol class.

Does not report protocol mismatch when concrete class has attribute with covariant type and protocol attribute is mutable.

Does not reject ClassVar in concrete class when attribute in protocol is not ClassVar.

Does not reject read-only property in concrete class when attribute in protocol is mutable.

Does not reject covariant attribute type when protocol attribute is mutable.

Does not detect protocol mismatch if concrete method is missing annotations.

Does not detect protocol mismatch if concrete method's parameters are keyword-only.

Does not detect protocol mismatch if concrete method's parameters are position-only.

Does not detect protocol mismatch if concrete method is a classmethod.

Does not detect protocol mismatch if concrete method is a staticmethod.

Does not reject read-only property in concrete class when protocol has settable property.

Does not reject immutable named tuple attribute in concrete class when protocol attribute is mutable.

Does not reject immutable frozen dataclass attribute in concrete class when protocol attribute is mutable.

     protocols_explicit
Pass*

Does not report unimplemented attributes for class that explicitly derives from protocol until it is instantiated.

Pass
Partial

Does not report error when calling unimplemented protocol method from derived class.

Does not report error when method is not implemented in derived class.

Partial

Reports errors for protocol static method with "..." implementation.

Does not report error when calling unimplemented protocol method from derived class.

Does not report type incompatibility when assigning to attribute defined in protocol.

Does not reject instantiation of class that derives from protocol but doesn't implement attribute.

Does not report instantiation of class that derives from protocol but doesn't implement method.

     protocols_generic
Partial

Fails protocol matching when method-scoped TypeVar is used in protocol.

Pass
Partial

Does not reject the use of Protocol and Generic together as base classes.

Does not detect protocol mismatch when method-scoped TypeVar is used in protocol.

Partial

Does not correctly enforce contravariance in protocol type compatibility tests.

Does not correctly enforce invariance in protocol type compatibility tests.

Does not detect protocol mismatch when method-scoped TypeVar is used in protocol.

     protocols_mergingPassPass
Partial

Does not reject a protocol class that derives from a non-protocol class.

Partial

Does not reject a protocol class that derives from a non-protocol class.

Does not report attempt to instantiate abstract class downgraded from protocol class.

     protocols_modulesPassPass
Unsupported

Does not perform protocol checks for modules.

Partial

Does not report incompatibilities for protocol methods.

     protocols_recursivePassPassPass
Partial

Incorrectly reports type error for some recursive protocols.

     protocols_runtime_checkable
Partial

Does not report unsafe overlap for runtime_checkable protocol.

Pass
Unsupported

Does not reject isinstance or issubclass call for protocol that is not runtime_checkable.

Does not reject issubclass call for data protocol.

Does not report unsafe overlap for runtime_checkable protocol.

Unsupported

Does not reject isinstance or issubclass call for protocol that is not runtime_checkable.

Does not reject issubclass call for data protocol.

Does not report unsafe overlap for runtime_checkable protocol.

     protocols_selfPassPassPass
Partial

Does not properly handle Self type within a protocol.

     protocols_subtypingPassPassPass
Partial

Does not reject attempt to instantiate protocol class.

Does not report some protocol type compatibility violations involving contravariance.

     protocols_variancePassPass
Unsupported

Does not detect incorrect TypeVar variance within generic protocols.

Unsupported

Does not detect incorrect TypeVar variance within generic protocols.

Callables
     callables_annotationPassPass
Partial

Does not evaluate correct type for `*args: int` parameter.

Does not reject illegal form `Callable[[...], int]`.

Pass
     callables_kwargsPassPass
Unsupported

Does not understand Unpack in the context of **kwargs annotation.

Unsupported

Does not understand Unpack in the context of **kwargs annotation.

     callables_protocolPassPass
Partial

Does not correctly handle callback protocol that declares attributes in all functions.

Does not report type incompatibility for callback protocol with positional-only parameters.

Incorrectly reports type compatibility error with callback that has *args and **kwargs.

Does not report type incompatibility for callback missing a default argument for positional parameter.

Does not report type incompatibility for callback missing a default argument for keyword parameter.

Unsupported

Does not properly handle type compatibility checks with callback protocols.

Overloads
     overloads_basicPassPass
Partial

Does not reject a function with a single @overload signature.

Partial

Does not reject a function with a single @overload signature.

Does not reject a function with @overload signature but no implementation.

Dataclasses
     dataclasses_descriptors
Partial

Does not correctly evaluate type of descriptor access.

Pass
Partial

Incorrectly generates error when calling constructor of dataclass with descriptor.

Unsupported

Does not understand descriptor objects in dataclass.

     dataclasses_frozenPassPass
Partial

Does not reject frozen dataclass inherited from non-frozen dataclass.

Does not reject non-frozen dataclass inherited from frozen dataclass.

Unsupported

Does not report assignment to field within frozen dataclass instance.

Does not reject frozen dataclass inherited from non-frozen dataclass.

Does not reject non-frozen dataclass inherited from frozen dataclass.

     dataclasses_hash
Partial

Does not report when dataclass is not compatible with Hashable protocol.

Pass
Partial

Does not report when dataclass is not compatible with Hashable protocol.

Partial

Does not report when dataclass is not compatible with Hashable protocol.

     dataclasses_inheritancePassPass
Partial

Does not reject ClassVar that is overridden by instance variable.

Does not reject instance variable that is overridden by ClassVar.

Partial

Does not reject ClassVar that is overridden by instance variable.

Does not reject instance variable that is overridden by ClassVar.

     dataclasses_kwonly
Partial

Incorrectly rejects kw_only field with default before positional field.

PassPass
Partial

Incorrectly reports error when kw_only field has default value.

Incorrectly rejects kw_only field with default before positional field.

     dataclasses_orderPassPass
Partial

Does not report type incompatibility with comparison operator.

Partial

Does not report type incompatibility with comparison operator.

     dataclasses_postinitPassPass
Unsupported

Does not perform validation of `__post_init__` method.

Does not reject access of `InitVar` from object.

Partial

Does not validate `__post_init__` method.

Reports incorrect error for incompatible `__post_init__` method override.

     dataclasses_slots
Partial

Does not reject write to instance variable that is not defined in __slots__.

Pass
Partial

Does not report error when `slots=True` is used with `__slots__` definition.

Does not reject write to instance variable that is not defined in __slots__.

Does not reject access to `__slots__` from dataclass instance when `slots=False`.

Partial

Does not report error when `slots=True` is used with `__slots__` definition.

Does not reject write to instance variable that is not defined in __slots__.

Incorrectly reports error when accessing `__slots__` when `slots=True`.

     dataclasses_transform_classPassPass
Unsupported

Does not understand @dataclass_transform.

Unsupported

Does not understand @dataclass_transform.

     dataclasses_transform_field
Partial

Does not properly handle field constructor that has default value for `kw_only` or `init` parameter.

Pass
Unsupported

Does not understand @dataclass_transform.

Unsupported

Does not understand @dataclass_transform.

     dataclasses_transform_func
Partial

Does not handle `kw_only=False` override when `kw_only_default=True`.

Does not report error when `order=False` and comparison operators are used.

Pass
Unsupported

Does not understand @dataclass_transform.

Unsupported

Does not understand @dataclass_transform.

     dataclasses_transform_metaPassPass
Unsupported

Does not understand @dataclass_transform.

Unsupported

Does not understand @dataclass_transform.

     dataclasses_usagePassPass
Partial

Does not report error when field with no default follows field with default.

Incorrectly reports error with InitVar that has default value.

Pass
Typed dictionaries
     typeddicts_alt_syntax
Pass*

Does not support keyword-argument form of alternative syntax (deprecated in 3.11).

Pass
Partial

Does not report when name of TypedDict doesn't match assigned identifier name.

Does not support keyword-argument form of alternative syntax (deprecated in 3.11).

Partial

Does not reject use of variable as second argument to `TypedDict` call.

Does not report when name of TypedDict doesn't match assigned identifier name.

Does not support keyword-argument form of alternative syntax (deprecated in 3.11).

     typeddicts_class_syntaxPassPass
Partial

Does not reject methods within TypedDict class.

Does not report when metaclass is provided.

Does not report when other keyword argument is provided.

Does not support generic TypedDict class.

Partial

Does not reject methods within TypedDict class.

Does not report when metaclass is provided.

Does not report when other keyword argument is provided.

     typeddicts_finalPassPass
Partial

Does not handle value with literal type as index to TypedDict object.

Pass
     typeddicts_inheritancePassPass
Partial

Does not reject TypedDict class that inherits from non-TypedDict class.

Pass
     typeddicts_operationsPassPassPass
Partial

Does not report type violation with TypedDict value assignment.

Does not report reference to unknown key in TypedDict.

Does not reject `clear` method on TypedDict with required keys.

Does not reject delete operation for required key in TypedDict.

     typeddicts_required
Partial

Does not support nesting of `Annotated` and `Required` or `NotRequired`.

Pass
Partial

Does not reject use of `Required` in function parameter annotation.

Does not reject nested use of `Required` in type annotation.

Does not support recursive TypedDict definitions.

Partial

Does not reject use of `Required` in non-TypedDict class.

Does not reject use of `Required` in function parameter annotation.

Does not reject nested use of `Required` in type annotation.

     typeddicts_type_consistencyPassPass
Partial

Does not reject assignment of TypedDict with missing key.

Does not return non-Optional value from `get` method for required key.

Does not properly handle nested TypedDicts.

Partial

Does not report some type violations for TypedDict type compatibility.

Incorrectly reports type violation in cases where there is none.

Does not report type incompatibility between TypedDict and `dict[str, Any]`.

     typeddicts_usagePassPass
Partial

Does not report errant use of TypedDict in `isinstance` call.

Does not reject use of TypedDict as TypeVar bound.

Partial

Does not report errant use of TypedDict in `isinstance` call.

Does not reject use of TypedDict as TypeVar bound.

Tuples
     tuples_type_compat
Partial

Does not support tuple narrowing based on `len()` type guard (optional).

Incorrectly narrows tuple based on sequence patterns.

Pass
Partial

Does not support some unpacked tuple forms.

Does not report type violation when assigning `tuple[int, ...]` to `tuple[int]`.

Does not support tuple narrowing based on `len()` type guard (optional).

Does not correctly evaluate `Sequence[Never]` for `tuple[()]`.

Partial

Does not support unpacked tuple forms.

Does not report type violation when assigning `tuple[int, ...]` to `tuple[int]`.

Does not support tuple narrowing based on `len()` type guard (optional).

     tuples_type_formPassPass
Partial

Does not reject some invalid tuple forms involving ellipsis.

Pass
     tuples_unpacked
Partial

"More than one unpack" error is missing a line number.

Pass
Partial

Rejects some legal tuple type forms involving unpack.

Does not reject some illegal tuple type forms involving unpack.

Unsupported

Does not support `typing.Unpack`.

Does not support unpacked tuples within a tuple type form.

Named tuples
     namedtuples_define_class
Partial

Does not reject override of named tuple attribute in child class.

Pass
Partial

Does not evaluate correct type for indexed named tuple instance with slice.

Does not report out-of-range index access with named tuple instance.

Does not reject named tuple element with no default value after one with a default.

Incorrectly rejects assignment of named tuple to a tuple with compatible type.

Does not reject attempt to use NamedTuple with multiple inheritance.

Partial

Incorrectly rejects valid index of named tuple instance when using a negative index.

Does not evaluate correct type for indexed named tuple instance with slice.

Does not reject named tuple element with no default value after one with a default.

Does not reject override of named tuple attribute in child class.

Evaluates incorrect type for named tuple entry with a generic type.

Does not reject incorrect argument type passed to specialized generic named tuple constructor.

Does not reject attempt to use NamedTuple with multiple inheritance.

     namedtuples_define_functional
Pass*

Does not handle illegal named tuple names the same as runtime.

Pass
Pass*

Does not reject duplicate field names in functional form.

Does not handle illegal named tuple names the same as runtime.

Does not support defaults in functional form.

Pass*

Does not handle illegal named tuple names the same as runtime.

Does not support defaults in functional form.

     namedtuples_type_compatPassPass
Partial

Rejects valid type compatibility between named tuple and tuple.

Pass
     namedtuples_usage
Partial

Does not reject attempt to delete named tuple field by name.

Pass
Partial

Does not report out-of-range index access with named tuple instance.

Does not reject attempt to delete named tuple field by name.

Does not reject attempt to delete named tuple field by index.

Incorrectly handles subclasses of named tuples that add more attributes.

Partial

Incorrectly rejects valid index of named tuple instance when using a negative index.

Does not report out-of-range index access with named tuple instance.

Does not reject attempt to overwrite named tuple entry by name.

Does not reject attempt to delete named tuple entry by name.

Type narrowing
     narrowing_typeguardPassPass
Partial

Does not support `tuple` in `assert_type` call.

Does not reject TypeGuard method with too few parameters.

Partial

Does not reject TypeGuard method with too few parameters.

Type checker directives
     directives_assert_typePassPass
Unsupported

Does not understand "assert_type".

Pass
     directives_castPassPassPass
Partial

Does not reject a call to "cast" with additional arguments.

     directives_no_type_check
Partial

Does not honor `@no_type_check` class decorator.

Does not reject invalid call of `@no_type_check` function.

Pass*

Does not honor `@no_type_check` class decorator.

Pass*

Does not honor @no_type_check decorator.

Pass*

Does not honor @no_type_check decorator.

     directives_reveal_typePassPass
Unsupported

Does not understand reveal_type call.

Partial

Does not reject call to reveal_type with zero arguments.

Does not reject call to reveal_type with too many arguments.

     directives_type_checkingPassPassPassPass
     directives_type_ignore
Partial

Does not honor "# type: ignore" comment if comment includes additional text.

PassPass
Partial

Does not honor "# type: ignore" comment if comment includes additional text.

     directives_type_ignore_file1PassPass
Unsupported

Does not support file-level `#type: ignore` comment.

Pass
     directives_type_ignore_file2PassPassPass
Partial

Does not ignore `# type: ignore` if it occurs after docstrings in the file.

     directives_version_platform
Pass*

Does not understand three-element form of sys.version checks.

Does not understand os.name checks.

Pass
Partial

Does not support sys.platform checks.

Does not support os.name checks.

Pass*

Does not understand three-element form of sys.version checks.

Does not understand os.name checks.

+
 
mypy 1.8.0
1.4sec
pyright 1.1.350
1.4sec
pyre 0.9.19
2.8sec
pytype 2024.02.13
31.6sec
Type annotations
     annotations_coroutinesPassPass
Partial

Does not evaluate correct type for async function.

Partial

Does not evaluate correct type for async function.

     annotations_forward_refs
Partial

Does not report error for a forward reference that is not enclosed in quotes.

Does not report error for use of quoted type with "|" operator (runtime error).

Incorrectly generates error for quoted type defined in class scope.

Does not treat triple-quoted forward reference annotation as implicitly parenthesized.

Pass
Partial

Does not report error for a forward reference that is not enclosed in quotes.

Does not report error for use of quoted type with "|" operator (runtime error).

Does not reject f-string in quoted type annotation.

Incorrectly generates error for quoted type defined in class scope.

Does not generate error for unquoted type defined in class scope.

Does not treat triple-quoted forward reference annotation as implicitly parenthesized.

Partial

Does not reject some illegal type expression forms when quoted.

Incorrectly generates error for quoted type defined in class scope.

Evaluates incorrect type for class variable annotated with quoted type expression.

Does not treat triple-quoted forward reference annotation as implicitly parenthesized.

     annotations_generators
Partial

Does not report incompatible Generator type in `yield from` statement.

Pass
Partial

Does not report invalid return type for generator when function implicitly returns None.

Incorrectly evaluates type of call to async generator.

Partial

Does not report invalid return type for generator when function implicitly returns None.

Reports invalid error when return type of generator is annotated as a compatible protocol.

Does not report type violation in `yield from` statement.

     annotations_methods
Pass*

Type evaluation differs from other type checkers because of ambiguity in the spec related to method bindings.

Pass*

Type evaluation differs from other type checkers because of ambiguity in the spec related to method bindings.

Pass*

Type evaluation differs from other type checkers because of ambiguity in the spec related to method bindings.

Pass*

Type evaluation differs from other type checkers because of ambiguity in the spec related to method bindings.

     annotations_typeexprPassPassPass
Partial

Does not reject call expressions in type annotation.

Does not reject call lambda expression in type annotation.

Does not reject list expression in type annotation.

Does not reject ternary expression in type annotation.

Does not reject f-string in type annotation.

Does not reject module in type annotation.

Special types in annotations
     specialtypes_anyPassPass
Partial

Does not treat missing type argument as Any in generic type.

Does not support Any as a base class.

Pass
     specialtypes_neverPassPass
Partial

Does not treat Never as compatible with all other types.

Unsupported

Does not understand NoReturn or Never.

     specialtypes_nonePassPass
Partial

Does not correctly handle type annotation type[None].

Partial

Does not detect type incompatibility between None and type[None].

Does not detect type incompatibility between None and incompatible protocol.

     specialtypes_promotionsPassPass
Partial

Does not reject use of attribute that is compatible only with float.

Pass
     specialtypes_type
Partial

Does not treat `type` same as `type[Any]` for assert_type.

Does not allow access to unknown attributes from object of type `type[Any]`.

Pass
Partial

Does not reject Callable when passed to type[T].

Does not treat `type` same as `type[Any]` for assert_type.

Does not allow access to unknown attributes from object of type `type[Any]`.

Does not reject access to unknown attributes from object of type `Type[object]`.

Reports type incompatibility between `type` and `Callable[..., Any]`.

Partial

Does not reject Callable when passed to type[T].

Does not allow access to known attributes from object of type `type[Any]`.

Generics
     generics_base_classPassPass
Pass*

Doesn't allow using generic in assert_type expression.

Partial

False negative on passing SymbolTable to dict[str, list[object]].

     generics_basicPassPass
Partial

False positives in examples using constrained type variables.

False negative in custom map example.

False positive using `iter`.

False negative for generic metaclass.

Partial

False positives in examples using constrained type variables.

False negative for generic metaclass.

     generics_paramspec_basic
Partial

Does not reject ParamSpec when used "bare" in type alias definition.

Pass
Partial

Does not enforce name consistency for ParamSpec assigned to identifier.

Incorrectly reports error for legitimate use of ParamSpec in generic type alias.

Does not reject ParamSpec when used in various invalid locations.

Unsupported

Does not support ParamSpec.

     generics_paramspec_components
Partial

Does not report illegal use of "P.args" on normal parameter.

Does not report error when P.args is specified but P.kwargs is missing.

Does not report error when P is out of scope and P.args and P.kwargs is used.

Does not report error when keyword argument is specified between P.args and P.kwargs.

Does not report error when calling callable and argument is missing for concatenated parameters.

Pass
Partial

Does not report illegal use of "P.args" on normal parameter.

Does not report error when P.args is specified but P.kwargs is missing.

Does not report error when P is out of scope and P.args and P.kwargs is used.

Does not report error when calling callback defined with ParamSpec with incorrect arguments.

Does not report error when keyword argument is specified between P.args and P.kwargs.

Does not report error when calling callable and argument is missing for concatenated parameters.

Unsupported

Does not support ParamSpec.

     generics_paramspec_semanticsPass
Pass*

Constraint solver doesn't find common type for two signatures captured by a single ParamSpec (allowed).

Partial

Constraint solver doesn't find common type for two signatures captured by a single ParamSpec (allowed).

Reports error for legitimate Callable type annotation that uses Concatenate.

Does not evaluate the correct type for call of Callable defined with Concatenate.

Unsupported

Does not support ParamSpec.

     generics_paramspec_specializationPassPass
Partial

Reports error for legitimate use of ParamSpec and Concatenate in function signature.

Reports error for legitimate specialization of generic class parameterized with ParamSpec.

Partial

Rejects valid specialization of ParamSpec using list expression.

Does not reject invalid specialization of class with both TypeVar and ParamSpec.

Reports error for valid method call involving ParamSpec.

     generics_scoping
Partial

False negative on generic class nested within generic class with same type variable.

Pass
Partial

False negative on generic class nested within generic function with same type variable.

False negative on generic class nested within generic class with same type variable.

Pass
     generics_self_advanced
Partial

Does not infer the type of an unannotated `self` parameter to be type `Self`.

Does not retain `Self` when calling method that returns `Self`.

Does not infer the type of an unannotated `cls` parameter to be type `type[Self]`.

Does not retain `Self` when accessing attribute through `type[Self]`.

Pass
Unsupported

Does not understand `Self` type.

Unsupported

Does not understand `Self` type.

     generics_self_attributesPassPass
Unsupported

Does not understand `Self` type.

Unsupported

Does not understand `Self` type.

     generics_self_basic
Partial

Does not properly handle constructor call through `cls` parameter.

Pass
Unsupported

Does not understand `Self` type.

Unsupported

Does not understand `Self` type.

     generics_self_protocolsPassPass
Partial

Does not reject protocol compatibility due to method `Self` return type.

Partial

Does not reject protocol compatibility due to method `Self` return type.

     generics_self_usagePassPass
Unsupported

Does not understand `Self` type.

Unsupported

Does not understand `Self` type.

     generics_syntax_compatibility
Unsupported

Type parameter syntax not yet support.

Pass
Unsupported

Type parameter syntax not yet support.

Unsupported

Type parameter syntax not yet support.

     generics_syntax_declarations
Unsupported

Type parameter syntax not yet support.

Pass
Unsupported

Type parameter syntax not yet support.

Unsupported

Type parameter syntax not yet support.

     generics_syntax_infer_variance
Unsupported

Type parameter syntax not yet support.

Pass
Unsupported

Type parameter syntax not yet support.

Unsupported

Type parameter syntax not yet support.

     generics_syntax_scoping
Unsupported

Type parameter syntax not yet support.

Pass
Unsupported

Type parameter syntax not yet support.

Unsupported

Type parameter syntax not yet support.

     generics_type_erasure
Partial

Infers Node[Never] instead of Node[Any] when argument is not provided.

False negative on instance attribute access on type(node).

Pass
Partial

Doesn't allow using Node[Any] in assert_type expression.

False negatives on instance attribute access on the type.

Unsupported
     generics_typevartuple_args
Partial

Does not enforce that tuples captured by TypeVarTuple are same type.

Pass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_basic
Partial

Does not enforce that tuples captured by TypeVarTuple are same length.

Does not enforce that tuples captured by TypeVarTuple are same type.

Does not enforce that tuples captured by TypeVarTuple are invariant in non-tuple class.

Pass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_callablePassPass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_concatPassPass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_overloadsPassPass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_specialization
Partial

Incorrectly specializes generic alias that includes a TypeVar and TypeVarTuple if no type arguments are provided.

Rejects specialization of generic type alias defined as a tuple containing a TypeVar.

"More than one Unpack" error message has no line number.

Pass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_typevartuple_unpack
Partial

Does not reject multiple unpack operators in a tuple.

Pass
Unsupported

Does not support TypeVarTuple.

Unsupported

Does not support TypeVarTuple.

     generics_variance
Partial

Does not reject use of class-scoped TypeVar used in a base class when variance is incompatible.

Pass
Partial

Does not reject a TypeVar that is defined as both covariant and contravariant.

Does not reject use of class-scoped TypeVar used in a base class when variance is incompatible.

Unsupported

Does not support covariant or contravariant TypeVars.

     generics_variance_inference
Unsupported

Type parameter syntax not yet support.

Pass
Unsupported

Type parameter syntax not yet support.

Unsupported

Type parameter syntax not yet support.

Type qualifiers
     qualifiers_annotated
Partial

Does not allow ClassVar to be nested within Annotated.

Does not allow Final to be nested within Annotated.

Does not allow Required and NotRequired to be nested within Annotated.

Does not reject type[T] compatibility for type alias defined with Annotated.

Does not reject call of type alias defined with Annotated.

Pass
Partial

Does not reject Annotated with a single parameter.

Does not reject call of Annotated with no type arguments.

Partial

Does not reject some illegal type expression forms used in Annotated.

Does not report type incompatibility between Annotated and type[T].

Does not reject call of Annotated.

Does not allow TypeVar to be used in type alias when wrapped with Annotated.

     qualifiers_final_annotation
Partial

Does not treat use of Final name as if it was replaced by the literal in NamedTuple definition.

Does not allow conditional assignment of Final instance variable in __init__ method.

Does not allow redefinition of private class variable that is marked Final in parent class.

Does not report modification of local Final variable via "for" statement.

Pass
Partial

Does not report Final variable with missing initialization in module scope.

Does not report error for invalid nesting of Final and ClassVar.

Does not treat use of Final name as if it was replaced by the literal in NamedTuple definition.

Partial

Does not report Final variable with missing initialization.

Does not reject Final instance variable declared outside of __init__ method.

Does not reject modification of global variable declared Final.

Does not reject modification of local variable declared Final.

     qualifiers_final_decoratorPassPass
Partial

Reports error for overloaded method implementation marked @final if its overloads do not.

Does not report error for overloaded @final method defined in stub file.

Reports misleading error when overload is marked @final but implementation is not.

Partial

Does not report error for overloaded @final method defined in stub file.

Does not report error for overload that is marked @final when implementation is not.

Class type compatibility
     classes_classvar
Partial

Internal error if TypeVarTuple is used in ClassVar.

Does not reject use of ParamSpec in ClassVar.

Rejects ClassVar nested in Annotated.

Does not reject use of ClassVar in TypeAlias definition.

Does not infer type of ClassVar from assignment if no type is provided.

Pass
Partial

Does not reject use of TypeVar in ClassVar.

Does not reject use of ParamSpec in ClassVar.

Does not reject use of ClassVar as a generic type argument.

Does not reject use of ClassVar in parameter type annotation.

Does not reject use of ClassVar in local variable annotation.

Does not reject use of ClassVar in instance variable annotation.

Does not reject use of ClassVar in return type annotation.

Does not reject use of ClassVar in type alias definition.

Partial

Does not reject use of TypeVar in ClassVar.

Does not reject use of ParamSpec in ClassVar.

Does not reject use of ClassVar as a generic type argument.

Rejects initialization of ClassVar if no type argument is provided.

Does not reject use of ClassVar in parameter type annotation.

Does not reject use of ClassVar in local variable annotation.

Does not reject use of ClassVar in instance variable annotation.

Does not reject use of ClassVar in return type annotation.

Does not reject use of ClassVar in type alias definition.

Does not reject assignment of ClassVar through instance of class.

     classes_override
Partial

Does not handle case where parent class derives from Any.

Pass
Unsupported

Does not yet support the @override decorator.

Unsupported

Does not yet support the @override decorator.

Type aliases
     aliases_explicit
Partial

Does not reject specialization of type alias that has already been implicitly specialized.

Pass
Partial

Incorrectly reports error for type alias defined with ParamSpec.

Incorrectly rejects some valid type aliases when used in annotations.

Incorrectly evaluates generic type alias with ParamSpec with missing type argument.

Does not report some illegal annotation forms as invalid type aliases.

Does not report invalid specialization of generic type aliases.

Incorrectly rejects import alias of `TypeAlias` when used to define type alias.

Does not report invalid specialization of already-specialized generic type alias.

Partial

Incorrectly reports error for type alias defined with ParamSpec.

Does not report invalid specialization of generic type alias with bound TypeVar.

Incorrectly evaluates generic type alias with ParamSpec with missing type argument.

Does not report some illegal annotation forms as invalid type aliases.

Does not report invalid specialization of already-specialized generic type alias.

     aliases_implicitPassPass
Partial

Incorrectly reports error for type alias defined with ParamSpec.

Incorrectly rejects some valid type aliases when used in annotations.

Incorrectly evaluates generic type alias with ParamSpec with missing type argument.

Does not report invalid specialization of generic type aliases.

Does not report error for attempt to instantiate union type alias.

Does not report invalid specialization of already-specialized generic type alias.

Partial

Incorrectly reports error for type alias defined with ParamSpec.

Does not report invalid specialization of generic type alias with bound TypeVar.

Incorrectly evaluates generic type alias with ParamSpec with missing type argument.

Allows some illegal annotation forms to be interpreted as valid type aliases.

Does not report invalid specialization of already-specialized generic type alias.

     aliases_newtypePassPass
Partial

Does not reject use of NewType in `isinstance` call.

Does not reject use of NewType in class definition statement.

Does not report inconsistency between name of NewType and assigned identifier name.

Does not reject use of NewType with generic class with TypeVar.

Does not reject use of NewType with protocol class.

Does not reject use of NewType with TypedDict class.

Does not reject use of NewType with Any.

Partial

Does not reject use of NewType in `isinstance` call.

Does not reject use of NewType in class definition statement.

Does not report inconsistency between name of NewType and assigned identifier name.

Does not reject use of NewType with generic class with TypeVar.

Does not reject use of NewType with protocol class.

Does not reject use of NewType with TypedDict class.

Does not reject use of NewType with Any.

     aliases_recursivePassPass
Partial

Does not properly handle some recursive type aliases.

Does not properly handle specialization of generic recursive type aliases.

Partial

Does not detect type violation for some deeply-nested types.

Does not properly handle `|` for unions in some recursive type alias definitions.

Does not detect cyclical references in recursive type alias definition.

     aliases_type_statement
Unsupported

Does not support `type` statement.

Pass
Unsupported

Does not support `type` statement.

Unsupported

Does not support `type` statement.

     aliases_typealiastype
Unsupported

Support for TypeAliasType is not implemented.

Pass
Unsupported

Support for TypeAliasType is not implemented.

Unsupported

Support for TypeAliasType is not implemented.

     aliases_variancePassPassPass
Unsupported

Does not detect variance incompatibility.

Literals
     literals_interactions
Partial

Does not narrow type of `x` with `x in Literal` type guard pattern.

Pass
Partial

Does not detect out-of-bound tuple literal index.

Does not narrow type of `x` with `x in Literal` type guard pattern.

Does not narrow type of `x` with `x == Literal` type guard pattern.

Partial

Incorrectly rejects some legal Literal annotations.

Does not reject some illegal Literal annotations.

Does not use Literal to distinguish overloads.

Does not narrow based on `x is Literal` type guard pattern.

Does not narrow based on `x == Literal` type guard pattern.

     literals_literalstring
Unsupported

Support for `LiteralString` is not implemented.

PassPass
Unsupported

Does not understand `LiteralString` special form.

     literals_parameterizations
Partial

Rejects integer Literal with unary `+` operator.

Does not reject tuple within Literal.

Pass
Partial

Does not support type aliases in Literal type expression.

Does not support nested Literal type expression.

Does not reject unary + operator in Literal type expression.

Does not reject tuple in Literal type expression.

Does not reject "bare" Literal in type expression.

Unsupported

Does not understand `Literal` type annotation.

     literals_semanticsPassPass
Partial

Does not reject augmented operation that modifies literal value.

Unsupported

Does not understand `Literal` type annotation.

Protocols
     protocols_class_objectsPassPass
Partial

Does not reject protocol class assigned to type[Proto].

Incorrectly reports some class objects as incompatible with a protocol.

Fails to report some class objects as incompatible with a protocol.

Partial

Does not reject protocol class assigned to type[Proto].

Incorrectly reports some class objects as incompatible with a protocol.

Fails to report some class objects as incompatible with a protocol.

     protocols_definition
Partial

Does not detect protocol mismatch if concrete method is missing annotations.

Does not detect protocol mismatch if concrete method's parameters are position-only.

Pass
Partial

Does not reject ClassVar in concrete class when attribute in protocol is not ClassVar or vice versa.

Does not reject read-only property in concrete class when attribute in protocol is mutable.

Does not reject covariant attribute type when protocol attribute is mutable.

Does not reject read-only property in concrete class when protocol has settable property.

Does not reject immutable named tuple attribute in concrete class when protocol attribute is mutable.

Does not reject immutable frozen dataclass attribute in concrete class when protocol attribute is mutable.

Partial

Reports errors for protocol static method with "..." implementation.

Does not report error when instance variable is set through "self" access in protocol class.

Does not report protocol mismatch when concrete class has attribute with covariant type and protocol attribute is mutable.

Does not reject ClassVar in concrete class when attribute in protocol is not ClassVar.

Does not reject read-only property in concrete class when attribute in protocol is mutable.

Does not reject covariant attribute type when protocol attribute is mutable.

Does not detect protocol mismatch if concrete method is missing annotations.

Does not detect protocol mismatch if concrete method's parameters are keyword-only.

Does not detect protocol mismatch if concrete method's parameters are position-only.

Does not detect protocol mismatch if concrete method is a classmethod.

Does not detect protocol mismatch if concrete method is a staticmethod.

Does not reject read-only property in concrete class when protocol has settable property.

Does not reject immutable named tuple attribute in concrete class when protocol attribute is mutable.

Does not reject immutable frozen dataclass attribute in concrete class when protocol attribute is mutable.

     protocols_explicit
Pass*

Does not report unimplemented attributes for class that explicitly derives from protocol until it is instantiated.

Pass
Partial

Does not report error when calling unimplemented protocol method from derived class.

Does not report error when method is not implemented in derived class.

Partial

Reports errors for protocol static method with "..." implementation.

Does not report error when calling unimplemented protocol method from derived class.

Does not report type incompatibility when assigning to attribute defined in protocol.

Does not reject instantiation of class that derives from protocol but doesn't implement attribute.

Does not report instantiation of class that derives from protocol but doesn't implement method.

     protocols_generic
Partial

Fails protocol matching when method-scoped TypeVar is used in protocol.

Pass
Partial

Does not reject the use of Protocol and Generic together as base classes.

Does not detect protocol mismatch when method-scoped TypeVar is used in protocol.

Partial

Does not correctly enforce contravariance in protocol type compatibility tests.

Does not correctly enforce invariance in protocol type compatibility tests.

Does not detect protocol mismatch when method-scoped TypeVar is used in protocol.

     protocols_mergingPassPass
Partial

Does not reject a protocol class that derives from a non-protocol class.

Partial

Does not reject a protocol class that derives from a non-protocol class.

Does not report attempt to instantiate abstract class downgraded from protocol class.

     protocols_modulesPassPass
Unsupported

Does not perform protocol checks for modules.

Partial

Does not report incompatibilities for protocol methods.

     protocols_recursivePassPassPass
Partial

Incorrectly reports type error for some recursive protocols.

     protocols_runtime_checkable
Partial

Does not report unsafe overlap for runtime_checkable protocol.

Pass
Unsupported

Does not reject isinstance or issubclass call for protocol that is not runtime_checkable.

Does not reject issubclass call for data protocol.

Does not report unsafe overlap for runtime_checkable protocol.

Unsupported

Does not reject isinstance or issubclass call for protocol that is not runtime_checkable.

Does not reject issubclass call for data protocol.

Does not report unsafe overlap for runtime_checkable protocol.

     protocols_selfPassPassPass
Partial

Does not properly handle Self type within a protocol.

     protocols_subtypingPassPassPass
Partial

Does not reject attempt to instantiate protocol class.

Does not report some protocol type compatibility violations involving contravariance.

     protocols_variancePassPass
Unsupported

Does not detect incorrect TypeVar variance within generic protocols.

Unsupported

Does not detect incorrect TypeVar variance within generic protocols.

Callables
     callables_annotationPassPass
Partial

Does not evaluate correct type for `*args: int` parameter.

Does not reject illegal form `Callable[[...], int]`.

Pass
     callables_kwargsPassPass
Unsupported

Does not understand Unpack in the context of **kwargs annotation.

Unsupported

Does not understand Unpack in the context of **kwargs annotation.

     callables_protocolPassPass
Partial

Does not correctly handle callback protocol that declares attributes in all functions.

Does not report type incompatibility for callback protocol with positional-only parameters.

Incorrectly reports type compatibility error with callback that has *args and **kwargs.

Does not report type incompatibility for callback missing a default argument for positional parameter.

Does not report type incompatibility for callback missing a default argument for keyword parameter.

Unsupported

Does not properly handle type compatibility checks with callback protocols.

Overloads
     overloads_basicPassPass
Partial

Does not reject a function with a single @overload signature.

Partial

Does not reject a function with a single @overload signature.

Does not reject a function with @overload signature but no implementation.

Dataclasses
     dataclasses_descriptors
Partial

Does not correctly evaluate type of descriptor access.

Pass
Partial

Incorrectly generates error when calling constructor of dataclass with descriptor.

Unsupported

Does not understand descriptor objects in dataclass.

     dataclasses_frozenPassPass
Partial

Does not reject frozen dataclass inherited from non-frozen dataclass.

Does not reject non-frozen dataclass inherited from frozen dataclass.

Unsupported

Does not report assignment to field within frozen dataclass instance.

Does not reject frozen dataclass inherited from non-frozen dataclass.

Does not reject non-frozen dataclass inherited from frozen dataclass.

     dataclasses_hash
Partial

Does not report when dataclass is not compatible with Hashable protocol.

Pass
Partial

Does not report when dataclass is not compatible with Hashable protocol.

Partial

Does not report when dataclass is not compatible with Hashable protocol.

     dataclasses_inheritancePassPass
Partial

Does not reject ClassVar that is overridden by instance variable.

Does not reject instance variable that is overridden by ClassVar.

Partial

Does not reject ClassVar that is overridden by instance variable.

Does not reject instance variable that is overridden by ClassVar.

     dataclasses_kwonly
Partial

Incorrectly rejects kw_only field with default before positional field.

PassPass
Partial

Incorrectly reports error when kw_only field has default value.

Incorrectly rejects kw_only field with default before positional field.

     dataclasses_orderPassPass
Partial

Does not report type incompatibility with comparison operator.

Partial

Does not report type incompatibility with comparison operator.

     dataclasses_postinitPassPass
Unsupported

Does not perform validation of `__post_init__` method.

Does not reject access of `InitVar` from object.

Partial

Does not validate `__post_init__` method.

Reports incorrect error for incompatible `__post_init__` method override.

     dataclasses_slots
Partial

Does not reject write to instance variable that is not defined in __slots__.

Pass
Partial

Does not report error when `slots=True` is used with `__slots__` definition.

Does not reject write to instance variable that is not defined in __slots__.

Does not reject access to `__slots__` from dataclass instance when `slots=False`.

Partial

Does not report error when `slots=True` is used with `__slots__` definition.

Does not reject write to instance variable that is not defined in __slots__.

Incorrectly reports error when accessing `__slots__` when `slots=True`.

     dataclasses_transform_classPassPass
Unsupported

Does not understand @dataclass_transform.

Unsupported

Does not understand @dataclass_transform.

     dataclasses_transform_field
Partial

Does not properly handle field constructor that has default value for `kw_only` or `init` parameter.

Pass
Unsupported

Does not understand @dataclass_transform.

Unsupported

Does not understand @dataclass_transform.

     dataclasses_transform_func
Partial

Does not handle `kw_only=False` override when `kw_only_default=True`.

Does not report error when `order=False` and comparison operators are used.

Pass
Unsupported

Does not understand @dataclass_transform.

Unsupported

Does not understand @dataclass_transform.

     dataclasses_transform_metaPassPass
Unsupported

Does not understand @dataclass_transform.

Unsupported

Does not understand @dataclass_transform.

     dataclasses_usagePassPass
Partial

Does not report error when field with no default follows field with default.

Incorrectly reports error with InitVar that has default value.

Pass
Typed dictionaries
     typeddicts_alt_syntax
Pass*

Does not support keyword-argument form of alternative syntax (deprecated in 3.11).

Pass
Partial

Does not report when name of TypedDict doesn't match assigned identifier name.

Does not support keyword-argument form of alternative syntax (deprecated in 3.11).

Partial

Does not reject use of variable as second argument to `TypedDict` call.

Does not report when name of TypedDict doesn't match assigned identifier name.

Does not support keyword-argument form of alternative syntax (deprecated in 3.11).

     typeddicts_class_syntaxPassPass
Partial

Does not reject methods within TypedDict class.

Does not report when metaclass is provided.

Does not report when other keyword argument is provided.

Does not support generic TypedDict class.

Partial

Does not reject methods within TypedDict class.

Does not report when metaclass is provided.

Does not report when other keyword argument is provided.

     typeddicts_finalPassPass
Partial

Does not handle value with literal type as index to TypedDict object.

Pass
     typeddicts_inheritancePassPass
Partial

Does not reject TypedDict class that inherits from non-TypedDict class.

Pass
     typeddicts_operationsPassPassPass
Partial

Does not report type violation with TypedDict value assignment.

Does not report reference to unknown key in TypedDict.

Does not reject `clear` method on TypedDict with required keys.

Does not reject delete operation for required key in TypedDict.

     typeddicts_required
Partial

Does not support nesting of `Annotated` and `Required` or `NotRequired`.

Pass
Partial

Does not reject use of `Required` in function parameter annotation.

Does not reject nested use of `Required` in type annotation.

Does not support recursive TypedDict definitions.

Partial

Does not reject use of `Required` in non-TypedDict class.

Does not reject use of `Required` in function parameter annotation.

Does not reject nested use of `Required` in type annotation.

     typeddicts_type_consistencyPassPass
Partial

Does not reject assignment of TypedDict with missing key.

Does not return non-Optional value from `get` method for required key.

Does not properly handle nested TypedDicts.

Partial

Does not report some type violations for TypedDict type compatibility.

Incorrectly reports type violation in cases where there is none.

Does not report type incompatibility between TypedDict and `dict[str, Any]`.

     typeddicts_usagePassPass
Partial

Does not report errant use of TypedDict in `isinstance` call.

Does not reject use of TypedDict as TypeVar bound.

Partial

Does not report errant use of TypedDict in `isinstance` call.

Does not reject use of TypedDict as TypeVar bound.

Tuples
     tuples_type_compat
Partial

Does not support tuple narrowing based on `len()` type guard (optional).

Incorrectly narrows tuple based on sequence patterns.

Pass
Partial

Does not support some unpacked tuple forms.

Does not report type violation when assigning `tuple[int, ...]` to `tuple[int]`.

Does not support tuple narrowing based on `len()` type guard (optional).

Does not correctly evaluate `Sequence[Never]` for `tuple[()]`.

Partial

Does not support unpacked tuple forms.

Does not report type violation when assigning `tuple[int, ...]` to `tuple[int]`.

Does not support tuple narrowing based on `len()` type guard (optional).

     tuples_type_formPassPass
Partial

Does not reject some invalid tuple forms involving ellipsis.

Pass
     tuples_unpacked
Partial

"More than one unpack" error is missing a line number.

Pass
Partial

Rejects some legal tuple type forms involving unpack.

Does not reject some illegal tuple type forms involving unpack.

Unsupported

Does not support `typing.Unpack`.

Does not support unpacked tuples within a tuple type form.

Named tuples
     namedtuples_define_class
Partial

Does not reject override of named tuple attribute in child class.

Pass
Partial

Does not evaluate correct type for indexed named tuple instance with slice.

Does not report out-of-range index access with named tuple instance.

Does not reject named tuple element with no default value after one with a default.

Incorrectly rejects assignment of named tuple to a tuple with compatible type.

Does not reject attempt to use NamedTuple with multiple inheritance.

Partial

Incorrectly rejects valid index of named tuple instance when using a negative index.

Does not evaluate correct type for indexed named tuple instance with slice.

Does not reject named tuple element with no default value after one with a default.

Does not reject override of named tuple attribute in child class.

Evaluates incorrect type for named tuple entry with a generic type.

Does not reject incorrect argument type passed to specialized generic named tuple constructor.

Does not reject attempt to use NamedTuple with multiple inheritance.

     namedtuples_define_functional
Pass*

Does not handle illegal named tuple names the same as runtime.

Pass
Pass*

Does not reject duplicate field names in functional form.

Does not handle illegal named tuple names the same as runtime.

Does not support defaults in functional form.

Pass*

Does not handle illegal named tuple names the same as runtime.

Does not support defaults in functional form.

     namedtuples_type_compatPassPass
Partial

Rejects valid type compatibility between named tuple and tuple.

Pass
     namedtuples_usage
Partial

Does not reject attempt to delete named tuple field by name.

Pass
Partial

Does not report out-of-range index access with named tuple instance.

Does not reject attempt to delete named tuple field by name.

Does not reject attempt to delete named tuple field by index.

Incorrectly handles subclasses of named tuples that add more attributes.

Partial

Incorrectly rejects valid index of named tuple instance when using a negative index.

Does not report out-of-range index access with named tuple instance.

Does not reject attempt to overwrite named tuple entry by name.

Does not reject attempt to delete named tuple entry by name.

Type narrowing
     narrowing_typeguardPassPass
Partial

Does not support `tuple` in `assert_type` call.

Does not reject TypeGuard method with too few parameters.

Partial

Does not reject TypeGuard method with too few parameters.

Type checker directives
     directives_assert_typePassPass
Unsupported

Does not understand "assert_type".

Pass
     directives_castPassPassPass
Partial

Does not reject a call to "cast" with additional arguments.

     directives_no_type_check
Partial

Does not honor `@no_type_check` class decorator.

Does not reject invalid call of `@no_type_check` function.

Pass*

Does not honor `@no_type_check` class decorator.

Pass*

Does not honor @no_type_check decorator.

Pass*

Does not honor @no_type_check decorator.

     directives_reveal_typePassPass
Unsupported

Does not understand reveal_type call.

Partial

Does not reject call to reveal_type with zero arguments.

Does not reject call to reveal_type with too many arguments.

     directives_type_checkingPassPassPassPass
     directives_type_ignore
Partial

Does not honor "# type: ignore" comment if comment includes additional text.

PassPass
Partial

Does not honor "# type: ignore" comment if comment includes additional text.

     directives_type_ignore_file1PassPass
Unsupported

Does not support file-level `#type: ignore` comment.

Pass
     directives_type_ignore_file2PassPassPass
Partial

Does not ignore `# type: ignore` if it occurs after docstrings in the file.

     directives_version_platform
Pass*

Does not understand three-element form of sys.version checks.

Does not understand os.name checks.

Pass
Partial

Does not support sys.platform checks.

Does not support os.name checks.

Pass*

Does not understand three-element form of sys.version checks.

Does not understand os.name checks.