From 76577455eefdc888e19abeb03f4b1566c19bb0e5 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 23 Sep 2024 17:31:36 -0700 Subject: [PATCH 1/4] PEP 749: Add a FAKE_GLOBALS_VALUE format And some other changes from feedback by Larry. --- peps/pep-0749.rst | 69 ++++++++++++++++++++++++++++++----------------- 1 file changed, 44 insertions(+), 25 deletions(-) diff --git a/peps/pep-0749.rst b/peps/pep-0749.rst index 6503ef832f1..c167248e87c 100644 --- a/peps/pep-0749.rst +++ b/peps/pep-0749.rst @@ -27,7 +27,9 @@ specification: * We specify the behavior of wrapper objects that provide annotations, such as :py:func:`classmethod` and code that uses :py:func:`functools.wraps`. * There will not be a code flag for marking ``__annotate__`` functions - that can be run in a "fake globals" environment. + that can be run in a "fake globals" environment. Instead, we add a fourth format, + ``FAKE_GLOBALS_VALUE``, to allow third-party implementors of annotate functions to + indicate what formats the support. * Setting the ``__annotations__`` attribute directly will not affect the ``__annotate__`` attribute. * We add functionality to allow evaluating type alias values and type parameter bounds and defaults (which were added by :pep:`695` and :pep:`696`) using PEP 649-like semantics. @@ -74,6 +76,9 @@ We suggest the following deprecation plan: - In Python 3.14, ``from __future__ import annotations`` will continue to work as it did before, converting annotations into strings. + - If the future import is active, the ``__annotate__`` function of objects with + annotations will return the annotations as strings when called with the ``VALUE`` + format, reflecting the behavior of ``__annotations__``. - Sometime after the last release that did not support :pep:`649` semantics (expected to be 3.13) reaches its end-of-life, ``from __future__ import annotations`` is deprecated. Compiling any code that uses the future import will emit a :py:exc:`DeprecationWarning`. This will @@ -194,7 +199,8 @@ The module will contain the following functionality: * ``Format``: an enum that contains the possible formats of annotations. This will replace the ``VALUE``, ``FORWARDREF``, and ``SOURCE`` formats in :pep:`649`. PEP 649 proposed to make these values global members of the :py:mod:`inspect` - module; we prefer to place them within an enum. + module; we prefer to place them within an enum. We propose to add a fourth format, + ``FAKE_GLOBALS_VALUE`` (see below). * ``ForwardRef``: a class representing a forward reference; it may be returned by ``get_annotations()`` when the format is ``FORWARDREF``. The existing class :py:class:`typing.ForwardRef` will become an alias of this class. Its members include: @@ -241,6 +247,9 @@ What should this module be called? Some ideas: and ``from __future__ import annotations`` in the same module. The use of a common word as the name will make the module harder to search for. There is a PyPI package :pypi:`annotations`, but it had only a single release in 2015 and looks abandoned. +- ``annotation`` (in the singular): Similar, but does not cause confusion with the future + import. There is an abandoned PyPI package :pypi:`annotation`, but it apparently never + released any artifacts. - ``annotools``: Analogous to :py:mod:`itertools` and :py:mod:`functools`, but "anno" is a less obvious abbreviation than "iter" or "func". As of this writing, there is no PyPI package with this name. @@ -551,8 +560,8 @@ This approach would also mean that accessing ``.__annotations__`` on an instance of an annotated class no longer works. While this behavior is not documented, it is a long-standing feature of Python and is relied upon by some users. -Remove code flag for marking ``__annotate__`` functions -======================================================= +Adding the ``FAKE_GLOBALS_VALUE`` format +======================================== :pep:`649` specifies: @@ -567,33 +576,42 @@ Remove code flag for marking ``__annotate__`` functions it's expected that only ``__annotate__`` methods generated by the Python compiler will set it. -We have not found a need for this mechanism during our work to -add :pep:`649` support to the standard library. While it is true -that custom ``__annotate__`` functions may not work well with the -"fake globals" environment, this technique is used only when the -``__annotate__`` function raises :py:exc:`NotImplementedError` to -signal that it does not support the requested format. However, -manually implemented ``__annotate__`` functions are likely to support -all three annotation formats; often, they will consist of a call to -``annotationlib.call_annotate_function`` plus some transformation of the -result. - -In addition, the proposed mechanism couples the implementation with +However, this mechanism couples the implementation with low-level details of the code object. The code object flags are CPython-specific and the documentation :py:ref:`explicitly warns ` against relying on the values. +Larry Hastings suggested an alternative approach that does not +rely on code flags: a fourth format, ``FAKE_GLOBALS_VALUE``. +Compiler-generated annotate functions would support only the +``VALUE`` and ``FAKE_GLOBALS_VALUE`` formats, both of which are +implemented identically. The standard library would use the +``FAKE_GLOBALS_VALUE`` format when invoking an annotate function +in one of the special "fake globals" environments. + +This approach is useful as a forward-compatible mechanism for +adding new annotation formats in the future. Users who manually +write annotate functions should raise ``NotImplementedError`` if +the ``FAKE_GLOBALS_VALUE`` format is requested, so the standard +library will not call the manually written annotate function with +"fake globals", which could have unpredictable results. + Specification ------------- -The standard library will use the "fake globals" technique on any -``__annotate__`` function that raises :py:exc:`NotImplementedError` -when the requested format is not supported. +An additional format, ``FAKE_GLOBALS_VALUE``, is added to the ``Format`` enum in the +``annotationlib`` module, with value equal to 4. Compiler-generated +annotate functions will support this format and return the same value as +they would return for the ``VALUE`` format. The standard library will pass +this format to the ``__annotate__`` function when it is called in a "fake globals" +environment, as used to implement the ``FORWARDREF`` and ``SOURCE`` formats. +All public functions in the ``annotationlib`` module that accept a format +argument will raise :py:exc:`NotImplementedError` if the format is ``FAKE_GLOBALS_VALUE``. -Third-party code that implements ``__annotate__`` functions should either -support all three annotation formats, or be prepared to handle the -"fake globals" environment. This should be mentioned in the data model -documentation for ``__annotate__``. +Third-party code that implements ``__annotate__`` functions should raise +:py:exc:`NotImplementedError` if the ``FAKE_GLOBALS_VALUE`` format is passed +and the function is not prepared to be run in a "fake globals" environment. +This should be mentioned in the data model documentation for ``__annotate__``. Effect of setting ``__annotations__`` ===================================== @@ -709,7 +727,7 @@ Signature of ``__annotate__`` functions ``__annotate__(format: int) -> dict`` However, using ``format`` as a parameter name could lead to collisions -if an annotation uses a class named ``format``. The parameter should be +if an annotation uses a symbol named ``format``. The parameter should be positional-only and have a name that cannot be a legal identifier in order to avoid this problem. @@ -858,7 +876,8 @@ initial decisions, but the overall design is still his. I thank Carl Meyer and Alex Waygood for feedback on early drafts of this PEP. Alex Waygood, Alyssa Coghlan, and David Ellis provided insightful feedback and suggestions on the -interaction between metaclasses and ``__annotations__``. +interaction between metaclasses and ``__annotations__``. Larry Hastings also provided useful +feedback on this PEP. Appendix ======== From aea52c7b95ee7f90a51d75796f750b009b3e2981 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 23 Sep 2024 17:37:56 -0700 Subject: [PATCH 2/4] Update peps/pep-0749.rst Co-authored-by: Carl Meyer --- peps/pep-0749.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0749.rst b/peps/pep-0749.rst index c167248e87c..b6c8eaf3aab 100644 --- a/peps/pep-0749.rst +++ b/peps/pep-0749.rst @@ -29,7 +29,7 @@ specification: * There will not be a code flag for marking ``__annotate__`` functions that can be run in a "fake globals" environment. Instead, we add a fourth format, ``FAKE_GLOBALS_VALUE``, to allow third-party implementors of annotate functions to - indicate what formats the support. + indicate what formats they support. * Setting the ``__annotations__`` attribute directly will not affect the ``__annotate__`` attribute. * We add functionality to allow evaluating type alias values and type parameter bounds and defaults (which were added by :pep:`695` and :pep:`696`) using PEP 649-like semantics. From 750b9e58eb10f6ba1d8fb48fed87ad313b08ed2a Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 23 Sep 2024 17:44:19 -0700 Subject: [PATCH 3/4] fix rendering --- peps/pep-0749.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/peps/pep-0749.rst b/peps/pep-0749.rst index b6c8eaf3aab..67d3fbf1a6c 100644 --- a/peps/pep-0749.rst +++ b/peps/pep-0749.rst @@ -76,9 +76,11 @@ We suggest the following deprecation plan: - In Python 3.14, ``from __future__ import annotations`` will continue to work as it did before, converting annotations into strings. + - If the future import is active, the ``__annotate__`` function of objects with annotations will return the annotations as strings when called with the ``VALUE`` format, reflecting the behavior of ``__annotations__``. + - Sometime after the last release that did not support :pep:`649` semantics (expected to be 3.13) reaches its end-of-life, ``from __future__ import annotations`` is deprecated. Compiling any code that uses the future import will emit a :py:exc:`DeprecationWarning`. This will From 390207b23e557c4ab749d14f368b2043504b8854 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 25 Sep 2024 12:09:09 -0700 Subject: [PATCH 4/4] Rename; set value to 2 as suggested by Larry --- peps/pep-0749.rst | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/peps/pep-0749.rst b/peps/pep-0749.rst index 67d3fbf1a6c..68bfcae07fe 100644 --- a/peps/pep-0749.rst +++ b/peps/pep-0749.rst @@ -28,7 +28,7 @@ specification: and code that uses :py:func:`functools.wraps`. * There will not be a code flag for marking ``__annotate__`` functions that can be run in a "fake globals" environment. Instead, we add a fourth format, - ``FAKE_GLOBALS_VALUE``, to allow third-party implementors of annotate functions to + ``VALUE_WITH_FAKE_GLOBALS``, to allow third-party implementors of annotate functions to indicate what formats they support. * Setting the ``__annotations__`` attribute directly will not affect the ``__annotate__`` attribute. * We add functionality to allow evaluating type alias values and type parameter bounds and defaults @@ -202,7 +202,7 @@ The module will contain the following functionality: replace the ``VALUE``, ``FORWARDREF``, and ``SOURCE`` formats in :pep:`649`. PEP 649 proposed to make these values global members of the :py:mod:`inspect` module; we prefer to place them within an enum. We propose to add a fourth format, - ``FAKE_GLOBALS_VALUE`` (see below). + ``VALUE_WITH_FAKE_GLOBALS`` (see below). * ``ForwardRef``: a class representing a forward reference; it may be returned by ``get_annotations()`` when the format is ``FORWARDREF``. The existing class :py:class:`typing.ForwardRef` will become an alias of this class. Its members include: @@ -562,8 +562,8 @@ This approach would also mean that accessing ``.__annotations__`` on an instance of an annotated class no longer works. While this behavior is not documented, it is a long-standing feature of Python and is relied upon by some users. -Adding the ``FAKE_GLOBALS_VALUE`` format -======================================== +Adding the ``VALUE_WITH_FAKE_GLOBALS`` format +============================================= :pep:`649` specifies: @@ -584,17 +584,17 @@ CPython-specific and the documentation :py:ref:`explicitly warns