diff --git a/docs/spec/annotations.rst b/docs/spec/annotations.rst index 0d4260fee..0b0ce6416 100644 --- a/docs/spec/annotations.rst +++ b/docs/spec/annotations.rst @@ -55,42 +55,166 @@ decorators ``@property``, ``@staticmethod`` and ``@classmethod``. .. _valid-types: -Valid type expression forms ---------------------------- - -Type hints may be built-in classes (including those defined in -standard library or third-party extension modules), abstract base -classes, types available in the ``types`` module, and user-defined -classes (including those defined in the standard library or -third-party modules). - -While annotations are normally the best format for type hints, -there are times when it is more appropriate to represent them -by a special comment, or in a separately distributed stub -file. (See below for examples.) +Type and annotation expressions +------------------------------- + +The terms *type expression* and *annotation expression* denote specific +subsets of Python expressions that are used in the type system. All +type expressions are also annotation expressions, but not all annotation +expressions are type expressions. + +.. _`type-expression`: + +A *type expression* is any expression that validly expresses a type. Type +expressions are always acceptable in annotations and also in various other +places. Specifically, type expressions are used in the following locations: + +* In a type annotation (always as part of an annotation expression) +* The first argument to :ref:`cast() ` +* The second argument to :ref:`assert_type() ` +* The bounds and constraints of a ``TypeVar`` (whether created through the + old syntax or the native syntax in Python 3.12) +* The definition of a type alias (whether created through the ``type`` statement, + the old assignment syntax, or the ``TypeAliasType`` constructor) +* The type arguments of a generic class (which may appear in a base class + or in a constructor call) +* The definitions of fields in the functional forms for creating + :ref:`TypedDict ` and :ref:`NamedTuple ` types +* The base type in the definition of a :ref:`NewType ` + +.. _`annotation-expression`: + +An *annotation expression* is an expression that is acceptable to use in +an annotation context (a function parameter annotation, function return +annotation, or variable annotation). Generally, an annotation expression +is a type expression, optionally surrounded by one or more :term:`type qualifiers ` +or by `Annotated`. Each type qualifier is valid only in some contexts. Note +that while annotation expressions are the only expressions valid as type +annotations in the type system, the Python language itself makes no such +restriction: any expression is allowed. Annotations must be valid expressions that evaluate without raising -exceptions at the time the function is defined (but see below for -forward references). - -Annotations should be kept simple or static analysis tools may not be -able to interpret the values. For example, dynamically computed types -are unlikely to be understood. (This is an -intentionally somewhat vague requirement; specific inclusions and -exclusions may be added in the future as warranted by the discussion.) - -In addition to the above, the following special constructs defined -below may be used: ``None``, ``Any``, ``Union``, ``Tuple``, -``Callable``, all ABCs and stand-ins for concrete classes exported -from ``typing`` (e.g. ``Sequence`` and ``Dict``), type variables, and -type aliases. +exceptions at the time the function is defined (but see :ref:`forward-references`). + +.. _`expression-grammar`: + +The following grammar describes the allowed elements of type and annotation expressions: + +.. productionlist:: expression-grammar + annotation_expression: '[' `annotation_expression` ']' + : | '[' `annotation_expression` ']' + : | '[' `annotation_expression`']' + : | '[' `annotation_expression`']' + : | '[' `annotation_expression`']' + : | '[' `annotation_expression` ']' + : | '[' `annotation_expression` ',' + : expression (',' expression)* ']' + : | + : (valid only in variable annotations) + : | `unpacked` + : (valid only for *args annotations) + : | '[' name ']' + : (where name refers to an in-scope TypedDict; + : valid only in **kwargs annotations) + : | `string_annotation` + : (must evaluate to a valid `annotation_expression`) + : | name '.' 'args' + : (where name must be an in-scope ParamSpec; + : valid only in *args annotations) + : | name '.' 'kwargs' + : (where name must be an in-scope ParamSpec; + : valid only in **kwargs annotations) + : | `type_expression` + type_expression: + : | + : (valid only in some contexts) + : | + : | + : | + : | + : | name + : (where name must refer to a valid in-scope class, + : type alias, or TypeVar) + : | name '[' (`maybe_unpacked` | `type_expression_list`) + : (',' (`maybe_unpacked` | `type_expression_list`))* ']' + : (the `type_expression_list` form is valid only when + : specializing a ParamSpec) + : | name '[' '(' ')' ']' + : (denoting specialization with an empty TypeVarTuple) + : | '[' expression (',' expression) ']' + : (see documentation for Literal for restrictions) + : | `type_expression` '|' `type_expression` + : | '[' `type_expression` ']' + : | '[' `type_expression` (',' `type_expression`)* ']' + : | '[' ']' + : | '[' name ']' + : (where name must refer to a valid in-scope class + : or TypeVar) + : | '[' '...' ',' `type_expression` ']' + : | '[' name ',' `type_expression` ']' + : (where name must be a valid in-scope ParamSpec) + : | '[' '[' (`type_expression` ',')+ + : (name | '...') ']' ',' `type_expression` ']' + : (where name must be a valid in-scope ParamSpec) + : | '[' '[' `maybe_unpacked` (',' `maybe_unpacked`)* + : ']' ',' `type_expression` ']' + : | `tuple_type_expression` + : | '[' `type_expression` ',' + : expression (',' expression)* ']' + : | '[' `type_expression` ']' + : (valid only in some contexts) + : | '[' `type_expression` ']' + : (valid only in some contexts) + : | `string_annotation` + : (must evaluate to a valid `type_expression`) + maybe_unpacked: `type_expression` | `unpacked` + unpacked: '*' `unpackable` + : | '[' `unpackable` ']' + unpackable: `tuple_type_expression`` + : | name + : (where name must refer to an in-scope TypeVarTuple) + tuple_type_expression: '[' '(' ')' ']' + : (representing an empty tuple) + : | '[' `type_expression` ',' '...' ']' + : (representing an arbitrary-length tuple) + : | '[' `maybe_unpacked` (',' `maybe_unpacked`)* ']' + string_annotation: string + : (must be a string literal that is parsable + : as Python code; see "String annotations") + type_expression_list: '[' `type_expression` (',' `type_expression`)* ']' + : | '[' ']' + +Notes: + +* The grammar assumes the code has already been parsed as Python code, and + loosely follows the structure of the AST. Syntactic details like comments + and whitespace are ignored. + +* ```` refers to a :term:`special form`. Most special forms must be imported + from :py:mod:`typing` or ``typing_extensions``, except for ``None``, ``InitVar``, + ``type``, and ``tuple``. The latter two have aliases in :py:mod:`typing`: :py:class:`typing.Type` + and :py:class:`typing.Tuple`. ``InitVar`` must be imported from :py:mod:`dataclasses`. + ``Callable`` may be imported from either :py:mod:`typing` or :py:mod:`collections.abc`. + Special forms may be aliased + (e.g., ``from typing import Literal as L``), and they may be referred to by a + qualified name (e.g., ``typing.Literal``). There are other special forms that are not + acceptable in any annotation or type expression, including ``Generic``, ``Protocol``, + and ``TypedDict``. + +* Any leaf denoted as ``name`` may also be a qualified name (i.e., ``module '.' name`` + or ``package '.' module '.' name``, with any level of nesting). + +* Comments in parentheses denote additional restrictions not expressed in the + grammar, or brief descriptions of the meaning of a construct. + +.. _ `string-annotations`: .. _`forward-references`: -Forward references +String annotations ------------------ -When a type hint contains names that have not been defined yet, that +When a type hint cannot be evaluated at runtime, that definition may be expressed as a string literal, to be resolved later. A situation where this occurs commonly is the definition of a @@ -118,7 +242,7 @@ same namespaces in which default arguments to the same function would be evaluated. Moreover, the expression should be parseable as a valid type hint, i.e., -it is constrained by the rules from the section on :ref:`valid-types`. +it is constrained by the rules from :ref:`the expression grammar `. If a triple quote is used, the string should be parsed as though it is implicitly surrounded by parentheses. This allows newline characters to be diff --git a/docs/spec/class-compat.rst b/docs/spec/class-compat.rst index 7bf77a2c9..b49a20348 100644 --- a/docs/spec/class-compat.rst +++ b/docs/spec/class-compat.rst @@ -10,10 +10,10 @@ Class type compatibility (Originally specified in :pep:`526`.) -A covariant type ``ClassVar[T_co]`` exists in the ``typing`` +A :term:`type qualifier` ``ClassVar[T]`` exists in the :py:mod:`typing` module. It accepts only a single argument that should be a valid type, and is used to annotate class variables that should not be set on class -instances. This restriction is ensured by static checkers, +instances. This restriction is enforced by static checkers, but not at runtime. Type annotations can be used to annotate class and instance variables @@ -26,7 +26,7 @@ in ``__init__`` or ``__new__``. The syntax is as follows:: damage: int # instance variable without default stats: ClassVar[dict[str, int]] = {} # class variable -Here ``ClassVar`` is a special class defined by the typing module that +Here ``ClassVar`` is a :term:`special form` defined by the :py:mod:`typing` module that indicates to the static type checker that this variable should not be set on instances. diff --git a/docs/spec/directives.rst b/docs/spec/directives.rst index 4657b1041..01c0f6bdb 100644 --- a/docs/spec/directives.rst +++ b/docs/spec/directives.rst @@ -18,6 +18,8 @@ should emit an error if the value is not of the specified type:: assert_type(name, str) # OK, inferred type of `name` is `str` assert_type(name, int) # type checker error +The second argument must be a valid :term:`type expression`. + .. _`reveal-type`: ``reveal_type()`` @@ -79,7 +81,8 @@ Some type checkers may not be able to infer that the type of ``a[index]`` is ``str`` and only infer ``object`` or ``Any``, but we know that (if the code gets to that point) it must be a string. The ``cast(t, x)`` call tells the type checker that we are confident that -the type of ``x`` is ``t``. At runtime a cast always returns the +the type of ``x`` is ``t``. ``t`` must be a valid :term:`type expression`. +At runtime a cast always returns the expression unchanged -- it does not check the type, and it does not convert or coerce the value. diff --git a/docs/spec/glossary.rst b/docs/spec/glossary.rst index 1b0f87346..39973ab6c 100644 --- a/docs/spec/glossary.rst +++ b/docs/spec/glossary.rst @@ -7,6 +7,11 @@ This section defines a few terms that may be used elsewhere in the specification .. glossary:: + annotation expression + An expression that is valid to use within an annotation. This is usually a + :term:`type expression`, sometimes with additional :term:`type qualifiers `. + See :ref:`"Type and annotation expression" ` for details. + distribution The packaged file which is used to publish and distribute a release. (:pep:`426`) @@ -25,6 +30,26 @@ This section defines a few terms that may be used elsewhere in the specification While most distributions are named after the one package they install, some distributions install multiple packages.) + special form + A special form is an object that has a special meaning within the type system, + comparable to a keyword in the language grammar. Examples include ``Any``, + ``Generic``, ``Literal``, and ``TypedDict``. Special forms can often but not always be used + within :ref:`type expressions `. Special forms can usually + be imported from the :py:mod:`typing` module or equivalently from ``typing_extensions``, + but some special forms are placed in other modules. + stub A file containing only type information, empty of runtime code (the filename ends in ``.pyi``). See :ref:`stub-files`. + + type expression + An expression that represents a type. The type system requires the use of type + expressions within :term:`annotation expression` and also in several other contexts. + See :ref:`"Type and annotation expression" ` for details. + + type qualifier + A type qualifier is a :term:`special form` that qualifies a :term:`type expression` to + form an :term:`annotation expression`. For example, the type qualifier :ref:`Final ` + can be used around a type to indicate that the annotated value may not be overridden or modified. + This term is also used for other special forms that modify a type, but using a different + syntactic context, such as the `@final ` decorator. diff --git a/docs/spec/historical.rst b/docs/spec/historical.rst index 54edf71d7..1c209a19b 100644 --- a/docs/spec/historical.rst +++ b/docs/spec/historical.rst @@ -280,10 +280,10 @@ and type checkers may warn if they are used. -------------------------- Before Python 3.10 (:pep:`604`), Python did not support the ``|`` operator -for creating unions of types. Therefore, the ``typing.Union`` special form can also +for creating unions of types. Therefore, the ``typing.Union`` :term:`special form` can also be used to create union types. Type checkers should treat the two forms as equivalent. -In addition, the ``Optional`` special form provides a shortcut for a union with ``None``. +In addition, the ``Optional`` :term:`special form` is equivalent to a union with ``None``. Examples: diff --git a/docs/spec/namedtuples.rst b/docs/spec/namedtuples.rst index 4c7fb74b6..835d196f4 100644 --- a/docs/spec/namedtuples.rst +++ b/docs/spec/namedtuples.rst @@ -1,3 +1,5 @@ +.. _`namedtuple`: + Named Tuples ============ diff --git a/docs/spec/narrowing.rst b/docs/spec/narrowing.rst index 51a3980a4..cbab74f09 100644 --- a/docs/spec/narrowing.rst +++ b/docs/spec/narrowing.rst @@ -13,14 +13,14 @@ TypeGuard (Originally specified in :pep:`647`.) -The symbol ``TypeGuard``, exported from the ``typing`` module, is a special form +The symbol ``TypeGuard``, exported from the ``typing`` module, is a :term:`special form` that accepts a single type argument. It is used to annotate the return type of a user-defined type guard function. Return statements within a type guard function should return bool values, and type checkers should verify that all return paths return a bool. ``TypeGuard`` is also valid as the return type of a callable, for example -in callback protocols and in the ``Callable`` special form. In these +in callback protocols and in the ``Callable`` :term:`special form`. In these contexts, it is treated as a subtype of bool. For example, ``Callable[..., TypeGuard[int]]`` is assignable to ``Callable[..., bool]``. @@ -115,7 +115,7 @@ TypeIs (Originally specified in :pep:`742`.) -The special form ``TypeIs`` is similar in usage, behavior, and runtime +The :term:`special form` ``TypeIs`` is similar in usage, behavior, and runtime implementation as ``TypeGuard``. ``TypeIs`` accepts a single type argument and can be used as the return type @@ -201,7 +201,7 @@ It is an error to narrow to a type that is not consistent with the input type:: ... ``TypeIs`` is also valid as the return type of a callable, for example -in callback protocols and in the ``Callable`` special form. In these +in callback protocols and in the ``Callable`` :term:`special form`. In these contexts, it is treated as a subtype of bool. For example, ``Callable[..., TypeIs[int]]`` is assignable to ``Callable[..., bool]``. diff --git a/docs/spec/protocol.rst b/docs/spec/protocol.rst index c07642445..5532276f0 100644 --- a/docs/spec/protocol.rst +++ b/docs/spec/protocol.rst @@ -41,7 +41,7 @@ protocol members. Defining a protocol ^^^^^^^^^^^^^^^^^^^ -Protocols are defined by including a special new class ``typing.Protocol`` +Protocols are defined by including a :term:`special form` ``typing.Protocol`` (an instance of ``abc.ABCMeta``) in the base classes list, typically at the end of the list. Here is a simple example:: diff --git a/docs/spec/qualifiers.rst b/docs/spec/qualifiers.rst index 223e9ed1c..49ec63f2d 100644 --- a/docs/spec/qualifiers.rst +++ b/docs/spec/qualifiers.rst @@ -3,6 +3,14 @@ Type qualifiers =============== +This chapter describes the behavior of some :term:`type qualifiers `. +Additional type qualifiers are covered in other chapters: + +* :ref:`ClassVar ` +* :ref:`NotRequired ` +* :ref:`ReadOnly ` +* :ref:`Required ` + .. _`at-final`: ``@final`` @@ -67,7 +75,7 @@ It is an error to use ``@final`` on a non-method function. (Originally specified in :pep:`591`.) -The ``typing.Final`` type qualifier is used to indicate that a +The ``typing.Final`` :term:`type qualifier` is used to indicate that a variable or attribute should not be reassigned, redefined, or overridden. Syntax @@ -247,7 +255,7 @@ details of the syntax: V == Annotated[list[tuple[int, int]], MaxLen(10)] -* As with most special forms, ``Annotated`` is not type compatible with +* As with most :term:`special forms `, ``Annotated`` is not type compatible with ``type`` or ``type[T]``:: v1: type[int] = Annotated[int, ""] # Type error diff --git a/docs/spec/special-types.rst b/docs/spec/special-types.rst index 76ac2cdec..b94222e60 100644 --- a/docs/spec/special-types.rst +++ b/docs/spec/special-types.rst @@ -62,7 +62,7 @@ equivalent to ``type(None)``. ``NoReturn`` ------------ -The ``typing`` module provides a special type ``NoReturn`` to annotate functions +The ``typing`` module provides a :term:`special form` ``NoReturn`` to annotate functions that never return normally. For example, a function that unconditionally raises an exception:: @@ -98,7 +98,7 @@ is unreachable and will behave accordingly:: ``Never`` --------- -Since Python 3.11, the ``typing`` module contains a primitive ``Never``. It +Since Python 3.11, the ``typing`` module contains a :term:`special form` ``Never``. It represents the bottom type, a type that has no members. The ``Never`` type is equivalent to ``NoReturn``, which is discussed above. @@ -165,7 +165,8 @@ type checker will infer the correct type of the result:: joe = new_user(BasicUser) # Inferred type is BasicUser The value corresponding to ``type[C]`` must be an actual class object -that's a subtype of ``C``, not a special form. In other words, in the +that's a subtype of ``C``, not a :term:`special form` or other kind of type. +In other words, in the above example calling e.g. ``new_user(BasicUser | ProUser)`` is rejected by the type checker (in addition to failing at runtime because you can't instantiate a union). diff --git a/docs/spec/tuples.rst b/docs/spec/tuples.rst index 96a28b76e..4f04b914a 100644 --- a/docs/spec/tuples.rst +++ b/docs/spec/tuples.rst @@ -78,7 +78,7 @@ An unpacked TypeVarTuple counts as an unbounded tuple in the context of this rul t6: tuple[*tuple[str, ...], *Ts] # Type error The ``*`` syntax requires Python 3.11 or newer. For older versions of Python, -the ``typing.Unpack`` special form can be used: +the ``typing.Unpack`` :term:`special form` can be used: ``tuple[int, Unpack[tuple[str, ...]], int]``. Unpacked tuples can also be used for ``*args`` parameters in a function diff --git a/docs/spec/typeddict.rst b/docs/spec/typeddict.rst index 6f536ebe4..5081e1ba1 100644 --- a/docs/spec/typeddict.rst +++ b/docs/spec/typeddict.rst @@ -533,7 +533,9 @@ types. In particular, they aren't subtypes of dictionary types. (Originally specified in :pep:`655`.) -The ``typing.Required`` type qualifier is used to indicate that a +.. _`required`: + +The ``typing.Required`` :term:`type qualifier` is used to indicate that a variable declared in a TypedDict definition is a required key: :: @@ -542,7 +544,9 @@ variable declared in a TypedDict definition is a required key: title: Required[str] year: int -Additionally the ``typing.NotRequired`` type qualifier is used to +.. _`notrequired`: + +Additionally the ``typing.NotRequired`` :term:`type qualifier` is used to indicate that a variable declared in a TypedDict definition is a potentially-missing key: @@ -645,10 +649,12 @@ Read-only Items (Originally specified in :pep:`705`.) +.. _`readonly`: + ``typing.ReadOnly`` type qualifier ---------------------------------- -The ``typing.ReadOnly`` type qualifier is used to indicate that an item declared in a ``TypedDict`` definition may not be mutated (added, modified, or removed):: +The ``typing.ReadOnly`` :term:`type qualifier` is used to indicate that an item declared in a ``TypedDict`` definition may not be mutated (added, modified, or removed):: from typing import ReadOnly