Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated spec and conformance test to reflect that a type variable tha… #1587

Merged
merged 3 commits into from
Jan 21, 2024
Merged

Updated spec and conformance test to reflect that a type variable tha… #1587

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3358d9b
Updated spec and conformance test to reflect that a type variable tha…
erictraut Jan 15, 2024
1b8c186
Updated proposed wording of change based on feedback.
erictraut Jan 20, 2024
929bf1c
Merge branch 'main' of https://github.com/python/typing into typevar_…
erictraut Jan 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 6 additions & 7 deletions conformance/results/mypy/generics_variance.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
conformant = "Partial"
notes = """
Does not reject a function-scoped TypeVar that is marked as covariant or contravariant.
Does not reject use of class-scoped TypeVar used in a base class when variance is incompatible.
"""
output = """
generics_variance.py:14: error: TypeVar cannot be both covariant and contravariant [misc]
generics_variance.py:78: error: Variance of TypeVar "T_co" incompatible with variance in parent type [type-var]
generics_variance.py:82: error: Variance of TypeVar "T_contra" incompatible with variance in parent type [type-var]
generics_variance.py:94: error: Variance of TypeVar "T_contra" incompatible with variance in parent type [type-var]
generics_variance.py:106: error: Variance of TypeVar "T_co" incompatible with variance in parent type [type-var]
generics_variance.py:126: error: Variance of TypeVar "T_co" incompatible with variance in parent type [type-var]
generics_variance.py:132: error: Variance of TypeVar "T_contra" incompatible with variance in parent type [type-var]
generics_variance.py:77: error: Variance of TypeVar "T_co" incompatible with variance in parent type [type-var]
generics_variance.py:81: error: Variance of TypeVar "T_contra" incompatible with variance in parent type [type-var]
generics_variance.py:93: error: Variance of TypeVar "T_contra" incompatible with variance in parent type [type-var]
generics_variance.py:105: error: Variance of TypeVar "T_co" incompatible with variance in parent type [type-var]
generics_variance.py:125: error: Variance of TypeVar "T_co" incompatible with variance in parent type [type-var]
generics_variance.py:131: error: Variance of TypeVar "T_contra" incompatible with variance in parent type [type-var]
"""
13 changes: 6 additions & 7 deletions conformance/results/pyre/generics_variance.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
conformant = "Partial"
notes = """
Does not reject a TypeVar that is defined as both covariant and contravariant.
Does not reject a function-scoped TypeVar that is marked as covariant or contravariant.
Does not reject use of class-scoped TypeVar used in a base class when variance is incompatible.
"""
output = """
generics_variance.py:78:0 Invalid type variance [46]: The type variable `Variable[T_co](covariant)` is incompatible with parent class type variable `Variable[T]` because subclasses cannot use more permissive type variables than their superclasses.
generics_variance.py:82:0 Invalid type variance [46]: The type variable `Variable[T_contra](contravariant)` is incompatible with parent class type variable `Variable[T]` because subclasses cannot use more permissive type variables than their superclasses.
generics_variance.py:94:0 Invalid type variance [46]: The type variable `Variable[T_contra](contravariant)` is incompatible with parent class type variable `Variable[T_co](covariant)` because subclasses cannot use more permissive type variables than their superclasses.
generics_variance.py:106:0 Invalid type variance [46]: The type variable `Variable[T_co](covariant)` is incompatible with parent class type variable `Variable[T_contra](contravariant)` because subclasses cannot use more permissive type variables than their superclasses.
generics_variance.py:126:0 Invalid type variance [46]: The type variable `Variable[T_co](covariant)` is incompatible with parent class type variable `Variable[T_contra](contravariant)` because subclasses cannot use more permissive type variables than their superclasses.
generics_variance.py:132:0 Invalid type variance [46]: The type variable `Variable[T_contra](contravariant)` is incompatible with parent class type variable `Variable[T_co](covariant)` because subclasses cannot use more permissive type variables than their superclasses.
generics_variance.py:77:0 Invalid type variance [46]: The type variable `Variable[T_co](covariant)` is incompatible with parent class type variable `Variable[T]` because subclasses cannot use more permissive type variables than their superclasses.
generics_variance.py:81:0 Invalid type variance [46]: The type variable `Variable[T_contra](contravariant)` is incompatible with parent class type variable `Variable[T]` because subclasses cannot use more permissive type variables than their superclasses.
generics_variance.py:93:0 Invalid type variance [46]: The type variable `Variable[T_contra](contravariant)` is incompatible with parent class type variable `Variable[T_co](covariant)` because subclasses cannot use more permissive type variables than their superclasses.
generics_variance.py:105:0 Invalid type variance [46]: The type variable `Variable[T_co](covariant)` is incompatible with parent class type variable `Variable[T_contra](contravariant)` because subclasses cannot use more permissive type variables than their superclasses.
generics_variance.py:125:0 Invalid type variance [46]: The type variable `Variable[T_co](covariant)` is incompatible with parent class type variable `Variable[T_contra](contravariant)` because subclasses cannot use more permissive type variables than their superclasses.
generics_variance.py:131:0 Invalid type variance [46]: The type variable `Variable[T_contra](contravariant)` is incompatible with parent class type variable `Variable[T_co](covariant)` because subclasses cannot use more permissive type variables than their superclasses.
"""
29 changes: 13 additions & 16 deletions conformance/results/pyright/generics_variance.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,31 +1,28 @@
conformant = "Partial"
notes = """
Does not reject a function-scoped TypeVar that is marked as covariant or contravariant.
"""
conformant = "Pass"
output = """
generics_variance.py:14:50 - error: TypeVar cannot be both covariant and contravariant
generics_variance.py:78:18 - error: Type "T_co@Class1" cannot be assigned to type variable "T@Inv"
generics_variance.py:77:18 - error: Type "T_co@Class1" cannot be assigned to type variable "T@Inv"
  Variance of type argument "T_co@Class1" is incompatible with base class "Inv" (reportGeneralTypeIssues)
generics_variance.py:82:18 - error: Type "T_contra@Class2" cannot be assigned to type variable "T@Inv"
generics_variance.py:81:18 - error: Type "T_contra@Class2" cannot be assigned to type variable "T@Inv"
  Variance of type argument "T_contra@Class2" is incompatible with base class "Inv" (reportGeneralTypeIssues)
generics_variance.py:94:20 - error: Type "T_contra@Co_Child3" cannot be assigned to type variable "T_co@Co"
generics_variance.py:93:20 - error: Type "T_contra@Co_Child3" cannot be assigned to type variable "T_co@Co"
  Variance of type argument "T_contra@Co_Child3" is incompatible with base class "Co" (reportGeneralTypeIssues)
generics_variance.py:106:28 - error: Type "T_co@Contra_Child3" cannot be assigned to type variable "T_contra@Contra"
generics_variance.py:105:28 - error: Type "T_co@Contra_Child3" cannot be assigned to type variable "T_contra@Contra"
  Variance of type argument "T_co@Contra_Child3" is incompatible with base class "Contra" (reportGeneralTypeIssues)
generics_variance.py:114:28 - error: Type "Co[T_co@Contra_Child5]" cannot be assigned to type variable "T_contra@Contra"
generics_variance.py:113:28 - error: Type "Co[T_co@Contra_Child5]" cannot be assigned to type variable "T_contra@Contra"
  Variance of type argument "Co[T_co@Contra_Child5]" is incompatible with base class "Contra" (reportGeneralTypeIssues)
generics_variance.py:127:20 - error: Type "T_co@CoContra_Child2" cannot be assigned to type variable "T_contra@CoContra"
generics_variance.py:126:20 - error: Type "T_co@CoContra_Child2" cannot be assigned to type variable "T_contra@CoContra"
  Variance of type argument "T_co@CoContra_Child2" is incompatible with base class "CoContra" (reportGeneralTypeIssues)
generics_variance.py:133:14 - error: Type "T_contra@CoContra_Child3" cannot be assigned to type variable "T_co@CoContra"
generics_variance.py:132:14 - error: Type "T_contra@CoContra_Child3" cannot be assigned to type variable "T_co@CoContra"
  Variance of type argument "T_contra@CoContra_Child3" is incompatible with base class "CoContra" (reportGeneralTypeIssues)
generics_variance.py:143:24 - error: Type "Co[T_co@CoContra_Child5]" cannot be assigned to type variable "T_contra@CoContra"
generics_variance.py:142:24 - error: Type "Co[T_co@CoContra_Child5]" cannot be assigned to type variable "T_contra@CoContra"
  Variance of type argument "Co[T_co@CoContra_Child5]" is incompatible with base class "CoContra" (reportGeneralTypeIssues)
generics_variance.py:164:33 - error: Type "Co[Contra[T_contra@CoToContraToContra]]" cannot be assigned to type variable "T_contra@Contra"
generics_variance.py:163:33 - error: Type "Co[Contra[T_contra@CoToContraToContra]]" cannot be assigned to type variable "T_contra@Contra"
  Variance of type argument "Co[Contra[T_contra@CoToContraToContra]]" is incompatible with base class "Contra" (reportGeneralTypeIssues)
generics_variance.py:168:37 - error: Type "Contra[Contra[T_co@ContraToContraToContra]]" cannot be assigned to type variable "T_contra@Contra"
generics_variance.py:167:37 - error: Type "Contra[Contra[T_co@ContraToContraToContra]]" cannot be assigned to type variable "T_contra@Contra"
  Variance of type argument "Contra[Contra[T_co@ContraToContraToContra]]" is incompatible with base class "Contra" (reportGeneralTypeIssues)
generics_variance.py:192:43 - error: Could not specialize type "Contra_TA[T_contra@Contra_TA]"
generics_variance.py:191:43 - error: Could not specialize type "Contra_TA[T_contra@Contra_TA]"
  Variance of type argument "Co_TA[Contra_TA[T_contra@CoToContraToContra_WithTA]]" is incompatible with "T_contra@Contra_TA"
generics_variance.py:197:15 - error: Could not specialize type "Contra_TA[T_contra@Contra_TA]"
generics_variance.py:196:15 - error: Could not specialize type "Contra_TA[T_contra@Contra_TA]"
  Variance of type argument "Contra_TA[Contra_TA[T_co@ContraToContraToContra_WithTA]]" is incompatible with "T_contra@Contra_TA"
"""
2 changes: 1 addition & 1 deletion conformance/results/results.html
View file
Open in desktop

Large diffs are not rendered by default.

7 changes: 3 additions & 4 deletions conformance/tests/generics_variance.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,9 @@ def dump_employee(e: E) -> E:
B_co = TypeVar("B_co", covariant=True)


# > Variance is only applicable to generic types; generic functions do not
# > have this property. The latter should be defined using only type variables
# > without covariant or contravariant keyword arguments.
def bad_func(x: list[B_co]) -> B_co: # Type checker error
# > Variance has no meaning, and should therefore be ignored by type checkers,
# > if a type variable is bound to a generic function or type alias.
def func(x: list[B_co]) -> B_co: # OK
...


Expand Down
31 changes: 10 additions & 21 deletions docs/spec/generics.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -562,32 +562,21 @@ mutable collection classes (e.g. ``MutableMapping`` and
a contravariant type is the ``Generator`` type, which is contravariant
in the ``send()`` argument type (see below).

Note: Covariance or contravariance is *not* a property of a type variable,
but a property of a generic class defined using this variable.
Variance is only applicable to generic types; generic functions
do not have this property. The latter should be defined using only
type variables without ``covariant`` or ``contravariant`` keyword arguments.
For example, the following example is
fine::
Variance is meaningful only when a type variable is bound to a generic class.
Variance has no meaning, and should therefore be ignored by type checkers, if
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be okay with this PR if you just remove "should therefore be ignored w by type checkers", so type checkers may choose to reject typevars declared as covariant or contravariant in contexts where variance has no meaning.

Suggested change
Variance has no meaning, and should therefore be ignored by type checkers, if
Variance has no meaning if

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If type checkers do not ignore variance when it has no meaning, what should they do instead? Keep in mind that every TypeVar (prior to PEP 695's "inferred variance") has an explicit variance; it's either invariant, covariant, or contravariant.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since "invariant" is declared by not adding {co,contra}variant=True to the TypeVar() call, I'd propose that type checkers accept invariant (i.e., default) typevars but issue an error for co/contra-variant typevars.

I believe it is reasonable to assume that omitting any mention of variance in the TypeVar() call is less explicit than explicitly using one of the two variance-declaring options (there is no invariant=True option).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was curious, so I quickly implemented a check in pyright for a TypeVar that's covariant or contravariant if it is scoped to a generic function or type alias. Here's the mypy_primer result: microsoft/pyright#6993. This isn't as noisy as I predicted. Interestingly, 100% of the violations it caught were in type aliases rather than functions.

I think we should make it clear in the spec that variance has no meaning for TypeVars scoped to functions or type aliases — and that type checkers should therefore ignore the variance for purposes of type checking. As for additional checks, I think we have three options:

  1. Indicate that type checkers should warn users if they attempt to use a covariant or contravariant TypeVar scoped to a generic function or type alias.
  2. Indicate that type checkers may warn users in this circumstance (but not mandate it).
  3. Indicate that type checkers should be silent about this circumstance, since variance is meaningless in this context.

I guess I don't have a strong opinion. The current draft text aligns to 3. It sounds like you prefer 1? Or 2?

I do think that mypy's current behavior is hard to defend. It emits an unrelated (and kind of meaningless) error if a covariant TypeVar is used for an input parameter or if a contravariant TypeVar is used for a return type. Since variance isn't meaningful in this case, those errors are inappropriate and confusing, IMO.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd go for (2), or (2a) don't allude to what type checkers should do at all.

(I'm beginning to realize that "has no meaning" is a rather ambiguous phrase, which implies different things in different contexts -- at least to me. I don't think it's the same as "is redundant" here.)

I agree that mypy's error is not what I thought it was and should be fixed independently.

a type variable is bound to a generic function or type alias.

from typing import TypeVar

class Employee: ...

class Manager(Employee): ...

E = TypeVar('E', bound=Employee)
For example::

def dump_employee(e: E) -> None: ...
T = TypeVar('T', covariant=True)

dump_employee(Manager()) # OK
class A(Generic[T]): # T is covariant in this context
...

while the following is prohibited::
def f(x: T) -> None: # Variance of T is meaningless in this context
...

B_co = TypeVar('B_co', covariant=True)

def bad_func(x: B_co) -> B_co: # Flagged as error by a type checker
...
Alias = list[T] | set[T] # Variance of T is meaningless in this context

.. _`paramspec`:

Expand Down
Loading