From baa0f092542c53ced2408e8a2a5b5ef1bd5dc09f Mon Sep 17 00:00:00 2001 From: "C.A.M. Gerlach" Date: Sun, 24 Jul 2022 22:26:08 -0500 Subject: [PATCH] Add PEP 517, PEP 518 & PEP 660 to specs and glossary --- source/discussions/wheel-vs-egg.rst | 2 +- source/glossary.rst | 203 +++++-- ...distributing-packages-using-setuptools.rst | 10 +- ...s-using-github-actions-ci-cd-workflows.rst | 6 +- source/guides/tool-recommendations.rst | 4 +- source/guides/using-manifest-in.rst | 6 +- source/key_projects.rst | 14 +- .../binary-distribution-format.rst | 6 + source/specifications/build-interface.rst | 559 ++++++++++++++++++ .../declaring-build-dependencies.rst | 254 +++++++- .../declaring-project-metadata.rst | 14 +- source/specifications/direct-url.rst | 8 +- source/specifications/index.rst | 12 +- .../pyproject-toml-config-file.rst | 75 +++ .../source-distribution-format.rst | 23 +- source/tutorials/installing-packages.rst | 15 +- source/tutorials/managing-dependencies.rst | 6 +- source/tutorials/packaging-projects.rst | 34 +- 18 files changed, 1118 insertions(+), 133 deletions(-) create mode 100644 source/specifications/build-interface.rst create mode 100644 source/specifications/pyproject-toml-config-file.rst diff --git a/source/discussions/wheel-vs-egg.rst b/source/discussions/wheel-vs-egg.rst index 8368fd2c0..0f0df3710 100644 --- a/source/discussions/wheel-vs-egg.rst +++ b/source/discussions/wheel-vs-egg.rst @@ -26,7 +26,7 @@ Here's a breakdown of the important differences between :term:`Wheel` and :term: * :term:`Wheel` archives do not include .pyc files. Therefore, when the distribution only contains Python files (i.e. no compiled extensions), and is compatible with Python 2 and 3, it's possible for a wheel to be "universal", - similar to an :term:`sdist `. + similar to an :term:`Sdist`. * :term:`Wheel` uses :pep:`PEP376-compliant <376>` ``.dist-info`` directories. Egg used ``.egg-info``. diff --git a/source/glossary.rst b/source/glossary.rst index 0d740bf39..a024839fd 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -12,19 +12,46 @@ Glossary extensions. + Build Frontend + + A tool that users interact with directly to trigger a build of + a :term:`Project`, which in turn invokes the project's + :term:`Build Backend` in a suitable environment + to generate a :term:`Built Distribution` (i.e. a :term:`Wheel`), + from a :term:`Source Tree` or :term:`Source Distribution`. + Examples include :ref:`build`, as well as :ref:`pip` + (when running a command such as ``pip wheel some-directory/``). + Compare to :term:`Integration Frontend`. + For more details, + see the :ref:`build-frontend-backend-interface` specification. + + + Build Backend + + A tool directly responsible for transforming a + :term:`Source Tree` or :term:`Source Distribution` + into a :term:`Built Distribution` (i.e. a :term:`Wheel`). + Typically invoked by a :term:`Build Frontend` rather than directly. + Examples include :ref:`flit`, :ref:`hatch` and :ref:`setuptools`. + For more details, + see the :ref:`build-frontend-backend-interface` specification. + + Built Distribution - A :term:`Distribution ` format containing files - and metadata that only need to be moved to the correct location on the - target system, to be installed. :term:`Wheel` is such a format, whereas - distutil's :term:`Source Distribution ` is not, in that it requires a build step before it can be - installed. This format does not imply that Python files have to be - precompiled (:term:`Wheel` intentionally does not include compiled - Python files). + A :term:`Distribution` format containing files and metadata + that only need to be moved to the correct location + on the target system to be installed. + :term:`Wheel` is such a format, whereas a :term:`Sdist` is not, + in that it requires processing by the :term:`Project`'s + :term:`Build Backend` before it can be installed. + This format does not imply that Python files have to be precompiled + (:term:`Wheel` intentionally does not include compiled Python files). Distribution Package + Distribution + Package A versioned archive file that contains Python :term:`packages `, :term:`modules `, and other resource files that are @@ -39,6 +66,27 @@ Glossary language distribution), which are often referred to with the single term "distribution". + + Editable Installation + Editable Mode + + An installation mode implying that the :term:`Source Tree` of the + :term:`project` being installed is available in a local directory, + in which users expect that changes to its *Python* code + become effective without the need of a new installation step. + + When a project is installed in "editable mode", + users expect it to behave as identically as practical + to a non-editable installation + (though some minor differences might be visible). + In particular, the code must be importable by other code, + and metadata must be available by standard mechanisms + such as ``importlib.metadata``. + + Formally specified in :pep:`660` and now defined in the + :ref:`build-frontend-backend-interface` specification. + + Egg A :term:`Built Distribution` format introduced by :ref:`setuptools`, @@ -75,6 +123,22 @@ Glossary is needed to prevent confusion with a :term:`Distribution Package` which is also commonly called a "package". + + Integration Frontend + + A tool that users run directly + that takes a set of :term:`Requirement`\s, + such as from a :term:`Project`'s :ref:`core-metadata`, + a :term:`Requirements File` or specified manually, + and attempts to update a working environment to satisfy them. + This may require locating, building and installing + a combination of :term:`Built Distribution`\s + and :term:`Source Distribution`\s, + including acting as a :term:`Build Frontend` in the latter case. + In a command like ``pip install lxml==2.4.0``, + :ref:`pip` is acting as an integration frontend. + + Module The basic unit of code reusability in Python, existing in one of two @@ -96,26 +160,30 @@ Glossary Project - A library, framework, script, plugin, application, or collection of data - or other resources, or some combination thereof that is intended to be - packaged into a :term:`Distribution `. - - Since most projects create :term:`Distributions ` - using either :pep:`518` ``build-system``, :ref:`distutils` or - :ref:`setuptools`, another practical way to define projects currently - is something that contains a :term:`pyproject.toml`, :term:`setup.py`, - or :term:`setup.cfg` file at the root of the project source directory. - - Python projects must have unique names, which are registered on - :term:`PyPI `. Each project will then - contain one or more :term:`Releases `, and each release may - comprise one or more :term:`distributions `. + A library, framework, script, plugin, application, + collection of data or other resources, or some combination thereof + that is intended to be packaged into a :term:`Distribution`. + + Since most projects create :term:`Distribution`\s + using a :term:`Build Backend` :ref:`declared ` + within a :ref:`pyproject.toml file `, + (or else implicitly use :ref:`setuptools`), + another practical way to define a project + is something that contains a :term:`pyproject.toml` + (or :term:`setup.py`/:term:`setup.cfg`) file + at the root of the project :term:`Source Tree`. + + Python projects must have unique :ref:`names `, + which are registered on a :term:`Package Index` + such as :term:`PyPI `. + Each project will contain one or more :term:`Releases `, + and each release may comprise one or more :term:`Distribution`\s. Note that there is a strong convention to name a project after the name - of the package that is imported to run that project. However, this - doesn't have to hold true. It's possible to install a distribution from - the project 'foo' and have it provide a package importable only as - 'bar'. + of the package that is imported to use that project. + However, this doesn't have to hold true. + It's possible to install a distribution from the project ``foo`` + and have it provide a package importable only as ``bar``. Pure Module @@ -149,10 +217,13 @@ Glossary domain name, ``pypi.python.org``, in 2017. It is powered by :ref:`warehouse`. + pyproject.toml - The tool-agnostic :term:`Project` specification file. - Defined in :pep:`518`. + The tool-agnostic :term:`Project` configuration file. + Originally introduced in :pep:`518` and now defined in the + :ref:`pyproject-toml-config-file` specification. + Release @@ -167,28 +238,33 @@ Glossary Requirement - A specification for a :term:`package ` to be - installed. :ref:`pip`, the :term:`PYPA ` recommended installer, allows various forms of specification - that can all be considered a "requirement". For more information, see the - :ref:`pip:pip install` reference. + A specification for a :term:`Package` + to be installed by an :term:`Integration Frontend`. + :ref:`pip`, the :term:`PyPA ` + recommended installer, + allows various forms of specification + that can all be considered a "requirement". + For more information, see the :ref:`pip:pip install` reference. Requirement Specifier - A format used by :ref:`pip` to install packages from a :term:`Package - Index`. For an EBNF diagram of the format, see the - `pkg_resources.Requirement - `_ - entry in the :ref:`setuptools` docs. For example, "foo>=1.3" is a - requirement specifier, where "foo" is the project name, and the ">=1.3" - portion is the :term:`Version Specifier` + A syntax used to declare the name and version of a :term:`Package` + that an :term:`Integration Frontend` such as :ref:`pip` + should install from a :term:`Package Index`. + For example, ``foo>=1.3`` is a requirement specifier, + where ``foo`` is the :ref:`project name ` + and ``>=1.3`` is the :term:`Version Specifier`. + The format was initially specified in :pep:`508`, + and is now defined in the :ref:`dependency-specifiers` specification. + Requirements File - A file containing a list of :term:`Requirements ` that can - be installed using :ref:`pip`. For more information, see the :ref:`pip` - docs on :ref:`pip:Requirements Files`. + A file containing a list of :term:`Requirement`\s that can + be installed using an :term:`Integration Frontend`, such as :ref:`pip`. + For more information, + see the :ref:`pip` docs on :ref:`pip:Requirements Files`. setup.py @@ -200,17 +276,32 @@ Glossary Source Archive - An archive containing the raw source code for a :term:`Release`, prior - to creation of a :term:`Source Distribution ` or :term:`Built Distribution`. + An archive containing the :term:`Source Tree` for a :term:`Release`, + prior to creation of a + :term:`Source Distribution` or :term:`Built Distribution`. + + + Source Distribution + Sdist + A :term:`Distribution` format + (generated using, e.g., ``python -m build --sdist``) + that provides metadata and the essential source files needed + by a :term:`Build Backend` to generate a :term:`Built Distribution` + for installation by an installer like :ref:`pip`. - Source Distribution (or "sdist") - A :term:`distribution ` format (usually generated - using ``python setup.py sdist``) that provides metadata and the - essential source files needed for installing by a tool like :ref:`pip`, - or for generating a :term:`Built Distribution`. + Source Tree + + A collection of files and directories (typically from a VCS checkout) + containing the raw source code of a :term:`project` + that is used for development. + Can be stored in a :term:`Source Archive` + and is used by a :term:`Build Backend` to generate a + :term:`Source Distribution` + and in turn a :term:`Built Distribution`, + as well as directly in an :term:`Editable Installation`. + Typically contains a :ref:`pyproject-toml-config-file` at its root. System Package @@ -235,11 +326,15 @@ Glossary wide. For more information, see the section on :ref:`Creating and using Virtual Environments`. + Wheel - A :term:`Built Distribution` format introduced by :pep:`427`, - which is intended to replace the :term:`Egg` format. Wheel is currently - supported by :ref:`pip`. + A :term:`Built Distribution` format, introduced by :pep:`427` + and now defined in the :ref:`binary-distribution-format` specification, + which replaces the legacy :term:`Egg` format. + Wheel is supported by :ref:`pip` and other installation tools, + and is the primary output of :term:`Build Backend`\s. + Working Set diff --git a/source/guides/distributing-packages-using-setuptools.rst b/source/guides/distributing-packages-using-setuptools.rst index c457a1e3b..2f8586f6e 100644 --- a/source/guides/distributing-packages-using-setuptools.rst +++ b/source/guides/distributing-packages-using-setuptools.rst @@ -670,7 +670,7 @@ Working in "development mode" You can install a project in "editable" or "develop" mode while you're working on it. When installed as editable, a project can be -edited in-place without reinstallation: +edited in-place without reinstallation: changes to Python source files in projects installed as editable will be reflected the next time an interpreter process is started. To install a Python package in "editable"/"development" mode @@ -683,10 +683,10 @@ Change directory to the root of the project directory and run: The pip command-line flag ``-e`` is short for ``--editable``, and ``.`` refers to the current working directory, so together, it means to install the current -directory (i.e. your project) in editable mode. This will also install any -dependencies declared with ``install_requires`` and any scripts declared with -``console_scripts``. Dependencies will be installed in the usual, non-editable -mode. +directory (i.e. your project) in :term:`Editable Mode`. +This will also install any dependencies declared with ``install_requires`` +and any scripts declared with ``console_scripts``. +Dependencies will be installed in the usual, non-editable mode. You may want to install some of your dependencies in editable mode as well. For example, supposing your project requires "foo" and "bar", but diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index 730098587..7454f75a9 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -95,8 +95,8 @@ install and activate Python 3.10. And now we can build dists from source. In this example, we'll use ``build`` package, assuming that your project has a -``pyproject.toml`` properly set up (see -:pep:`517`/:pep:`518`). +:term:`pyproject.toml` properly set up +(see :ref:`pyproject-toml-config-file` for full details). .. tip:: @@ -134,7 +134,7 @@ Now, whenever you push a tagged commit to your Git repository remote on GitHub, this workflow will publish it to PyPI. And it'll publish any push to TestPyPI which is useful for providing test builds to your alpha users as well as making -sure that your release pipeline remains healthy! +sure that your release pipeline remains healthy! .. _API token: https://pypi.org/help/#apitoken diff --git a/source/guides/tool-recommendations.rst b/source/guides/tool-recommendations.rst index fc0afd1df..3237ac9b6 100644 --- a/source/guides/tool-recommendations.rst +++ b/source/guides/tool-recommendations.rst @@ -49,8 +49,8 @@ Packaging tool recommendations * Use :ref:`setuptools` to define projects. [5]_ [6]_ -* Use :ref:`build` to create :term:`Source Distributions - ` and :term:`wheels `. +* Use :ref:`build` to create :term:`Source Distribution`\s + and :term:`Wheel`\s. If you have binary extensions and want to distribute wheels for multiple platforms, use :ref:`cibuildwheel` as part of your CI setup to build diff --git a/source/guides/using-manifest-in.rst b/source/guides/using-manifest-in.rst index 1eaf42c18..a08ef78de 100644 --- a/source/guides/using-manifest-in.rst +++ b/source/guides/using-manifest-in.rst @@ -4,8 +4,8 @@ Including files in source distributions with ``MANIFEST.in`` ============================================================ -When building a :term:`source distribution ` -for your package, by default only a minimal set of files are included. You may +When building a :term:`Source Distribution` for your package, +by default only a minimal set of files are included. You may find yourself wanting to include extra files in the source distribution, such as an authors/contributors file, a :file:`docs/` directory, or a directory of data files used for testing purposes. There may even be extra files that you @@ -72,7 +72,7 @@ Command Description (Files must be given as paths relative to the root of the project) :samp:`recursive-include {dir-pattern} {pat1} {pat2} ...` Add all files under directories matching ``dir-pattern`` that match any of the listed patterns :samp:`recursive-exclude {dir-pattern} {pat1} {pat2} ...` Remove all files under directories matching ``dir-pattern`` that match any of the listed patterns -:samp:`global-include {pat1} {pat2} ...` Add all files anywhere in the source tree matching any of the listed patterns +:samp:`global-include {pat1} {pat2} ...` Add all files anywhere in the :term:`Source Tree` matching any of the listed patterns :samp:`global-exclude {pat1} {pat2} ...` Remove all files anywhere in the source tree matching any of the listed patterns :samp:`graft {dir-pattern}` Add all files under directories matching ``dir-pattern`` :samp:`prune {dir-pattern}` Remove all files under directories matching ``dir-pattern`` diff --git a/source/key_projects.rst b/source/key_projects.rst index d404cbcef..3899f5794 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -39,8 +39,8 @@ build `GitHub `__ | `PyPI `__ -``build`` is a :pep:`517` compatible Python package builder. It provides a CLI to -build packages, as well as a Python API. +``build`` is a :ref:`build-interface`-compatible Python package builder. +It provides a CLI to build packages, as well as a Python API. .. _cibuildwheel: @@ -98,7 +98,8 @@ modules to PyPI. It focuses on `making the easy things easy `_ for packaging. Flit can generate a configuration file to quickly set up a simple project, build source distributions and wheels, and upload them to PyPI. -Flit uses ``pyproject.toml`` to configure a project. Flit does not rely on tools +Flit uses :term:`pyproject.toml` to configure a project. +Flit does not rely on tools such as :ref:`setuptools` to build distributions, or :ref:`twine` to upload them to PyPI. Flit requires Python 3, but you can use it to distribute modules for Python 2, so long as they can be imported on Python 3. @@ -116,7 +117,7 @@ hatch Hatch is a unified command-line tool meant to conveniently manage dependencies and environment isolation for Python developers. Python -package developers use Hatch and its build backend Hatchling to +package developers use Hatch and its :term:`Build Backend` Hatchling to configure, version, specify dependencies for, and publish packages to PyPI. Its plugin system allows for easily extending functionality. @@ -474,7 +475,8 @@ pdm PDM is a modern Python package manager with :pep:`582` support. It installs and manages packages in a similar way to ``npm`` that doesn't need to create a -:term:`virtual environment` at all. It also uses :term:`pyproject.toml` to store +:term:`virtual environment` at all. +It also uses :term:`pyproject.toml` to store project metadata as defined in :pep:`621`. .. _pex: @@ -535,7 +537,7 @@ poetry poetry is a command-line tool to handle dependency installation and isolation as well as building and packaging of Python packages. It -uses ``pyproject.toml`` and, instead of depending on the resolver +uses :term:`pyproject.toml` and, instead of depending on the resolver functionality within :ref:`pip`, provides its own dependency resolver. It attempts to speed users' experience of installation and dependency resolution by locally caching metadata about dependencies. diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index 62cee7d1a..e7b56a21e 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -114,6 +114,8 @@ Place ``.dist-info`` at the end of the archive. File Format ----------- +.. _wheel-file-name-convention: + File name convention '''''''''''''''''''' @@ -235,6 +237,8 @@ The layout offers a superset of the functionality provided by the existing wininst and egg binary formats. +.. _wheel-dist-info-directory: + The .dist-info directory ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -277,6 +281,8 @@ documentation and so forth from the distribution. During installation the contents of these subdirectories are moved onto their destination paths. +.. _wheel-signed-wheel-files: + Signed wheel files ------------------ diff --git a/source/specifications/build-interface.rst b/source/specifications/build-interface.rst new file mode 100644 index 000000000..6ea4fcc68 --- /dev/null +++ b/source/specifications/build-interface.rst @@ -0,0 +1,559 @@ +.. _build-frontend-backend-interface: +.. _build-interface: + +================================ +Build frontend-backend interface +================================ + +This specification defines a standard interface +between :term:`Build Frontend`\s and :term:`Build Backend`\s +to generate :term:`Distribution Package` artifacts +from :term:`Source Tree`\s of Python :term:`Project`\s. +It was originally introduced in :pep:`PEP 517 <517#build-backend-interface>` +and revised to include editable support in :pep:`660`. + +The build backend Python object specified in the +:ref:`build-system-build-backend-key` of the :ref:`build-system-table` +in the :term:`pyproject.toml` config file +MUST have attributes which provide some or all of the following hooks +(as Python callables) +that can be invoked by the build frontend. +The :ref:`build-interface-build-wheel` and :ref:`build-interface-build-sdist` +hooks MUST be provided by any build backend implementing this specification; +all other listed hooks ore optional. +The common ``config_settings`` argument is +:ref:`described after the individual hooks `. + + +.. _build-interface-wheel-hooks: + +Wheel hooks +=========== + +The following backend hooks relate to building a +:ref:`Wheel ` :term:`Built Distribution`. + + +.. _build-interface-build-wheel: + +build_wheel +----------- + +.. code-block:: python + + def build_wheel(wheel_directory, config_settings=None, metadata_directory=None): + ... + +*Mandatory hook*. +MUST build a :term:`Wheel` ``.whl`` file, +as defined in the :ref:`Wheel specification `, +and place it in the specified ``wheel_directory``. +MUST return the basename (not the full path) of the ``.whl`` file it creates +as a string. + +If the :term:`Build Frontend` has previously called +:ref:`build-interface-prepare-metadata-for-build-wheel` +and depends on the wheel resulting from this call +to have metadata matching this earlier call, +then it SHOULD provide the path to the created :file:`.dist-info` directory +as the ``metadata_directory`` argument. +If this argument is provided, +then ``build_wheel`` MUST produce a wheel with identical metadata. +The directory passed in by the build frontend MUST be identical +to the directory created by +:ref:`build-interface-prepare-metadata-for-build-wheel`, +including any unrecognized files it created. + +:term:`Build Backend`\s which do not provide +the :ref:`build-interface-prepare-metadata-for-build-wheel` hook +MAY either silently ignore +the ``metadata_directory`` parameter to ``build_wheel``, +or else raise an exception when it is set to anything other than ``None``. + +To ensure that wheels from different sources are built the same way, +frontends MAY call :ref:`build-interface-build-sdist` first, +and then call ``build_wheel`` in the unpacked :term:`Sdist`. +However, if the backend +:ref:`indicates that it is missing some requirements for creating an sdist +`, +the frontend SHOULD fall back to calling ``build_wheel`` +in the :term:`Source Tree`. + +The source tree MAY be read-only. +Backends SHOULD therefore be prepared to build without creating or modifying +any files in the source tree, but they MAY opt not to handle this situation, +in which case failures will be visible to the user. +Frontends are not responsible for any special handling +of read-only source directories. + +The backend MAY store intermediate artifacts +in cache locations or temporary directories. +The presence or absence of any caches SHOULD not make a material difference +to the final result of the build. + + +.. _build-interface-get-requires-for-build-wheel: + +get_requires_for_build_wheel +---------------------------- + +.. code-block:: python + + def get_requires_for_build_wheel(config_settings=None): + ... + +*Optional hook*. +This hook MUST return a list of strings containing +:ref:`dependency-specifiers`, +above and beyond those specified in the +:ref:`build-system.requires key ` +of :ref:`pyproject-toml-config-file`, +to be installed when calling the +:ref:`build-interface-build-wheel` +or :ref:`build-interface-prepare-metadata-for-build-wheel` hooks. + +Example: + +.. code-block:: python + + def get_requires_for_build_wheel(config_settings): + return ["wheel >= 0.25", "setuptools"] + +If not defined by the :term:`Build Backend`, +the default implementation is equivalent to ``return []``. + + +.. _build-interface-prepare-metadata-for-build-wheel: + +prepare_metadata_for_build_wheel +-------------------------------- + +.. code-block:: python + + def prepare_metadata_for_build_wheel(metadata_directory, config_settings=None): + ... + +*Optional hook*. +MUST create a :file:`.dist-info` directory containing :term:`Wheel` metadata +inside the specified ``metadata_directory``; +i.e., a directory like +:file:`{metadata_directory}/{package}-{version}.dist-info/`. +This MUST be a valid :file:`.dist-info` directory +as :ref:`defined in the wheel specification `, +except that it need not contain ``RECORD`` +or :ref:`signatures `. +The hook MAY also create other files inside this directory, +and a :term:`Build Frontend` MUST preserve, but otherwise ignore, such files; +the intention here is that in cases where +the metadata depends on build-time decisions, +the :term:`Build Backend` may need to record these decisions +in some convenient format for re-use by the actual wheel-building step. + +This MUST return the basename (not the full path) +of the :file:`.dist-info` directory it creates as a string. + +If a build frontend needs this information and the method is not defined, +it SHOULD call :ref:`build-interface-build-wheel` +and look at the resulting metadata directly. + + +.. _build-interface-sdist-hooks: + +Sdist hooks +=========== + +The following backend hooks relate to building a +:ref:`Sdist ` :term:`Source Distribution`. + + +.. _build-interface-build-sdist: + +build_sdist +----------- + +.. code-block:: python + + def build_sdist(sdist_directory, config_settings=None): + ... + +*Mandatory hook*. +MUST build a :term:`Sdist` :term:`Source Distribution`, +as defined in the :ref:`Sdist specification `, +and place it in the specified ``sdist_directory``. +MUST return the basename (not the full path) of the sdist file it creates +as a string. + +.. _build-interface-build-sdist-requirements: + +:term:`Build Frontend`\s MAY prefer produce wheels +from intermediate sdists, to ensure consistency. +However, some :term:`Build Backend`\s +MAY have extra requirements for creating sdists, +such as version control tools. +If the backend cannot produce an sdist because a dependency is missing, +or for another well understood reason, +it SHOULD raise an exception of a specific type +which it makes available as ``UnsupportedOperation`` on the backend object. +If the frontend gets this exception while building an sdist +as an intermediate for a wheel, +it SHOULD fall back to building a wheel directly. +The backend does not need to define this exception type +if it would never raise it. + + +.. _build-interface-get-requires-for-build-sdist: + +get_requires_for_build_sdist +---------------------------- + +.. code-block:: python + + def get_requires_for_build_sdist(config_settings=None): + ... + +*Optional hook*. +This hook MUST return a list of strings containing +:ref:`dependency-specifiers`, +above and beyond those specified in the +:ref:`build-system.requires key ` +of :ref:`pyproject-toml-config-file`, +to be installed when calling the :ref:`build-interface-build-sdist` hook. + +If not defined, the default implementation is equivalent to ``return []``. + + +.. _build-interface-editable-hooks: + +Editable hooks +============== + +The following backend hooks relate to building +an :term:`Editable Installation`. +These hooks are used to build a :term:`Wheel` that, when installed, +allows that :term:`Distribution` to be imported +from its :term:`Source Tree` directory. + + +.. _build-interface-build-editable: + +build_editable +-------------- + +.. code-block:: python + + def build_editable(wheel_directory, config_settings=None, metadata_directory=None): + ... + +*Optional hook*. +MUST build a :term:`Wheel` ``.whl`` file, +as defined in the :ref:`Wheel specification `, +and place it in the specified ``wheel_directory``. +MUST return the basename (not the full path) of the ``.whl`` file it creates +as a string. + +:term:`Build backend`\s MUST populate the generated wheel with files that, +when installed, will result in a working :term:`Editable Installation`. +Backends MAY use various techniques to achieve this goal, +such as :pep:`those suggested in PEP 660 <660#what-to-put-in-the-wheel>`. + +Backends MAY do an in-place build of the distribution as a side effect +so that any extension modules or other built artifacts are ready to be used. + +Runtime dependencies (:ref:`Requires-Dist `) +and other :ref:`core metadata ` of the built wheel +MUST be identical to that produced by :ref:`build-interface-build-wheel` +or :ref:`build-interface-prepare-metadata-for-build-wheel`; +with the exception that for ``build_editable``, +Build Backends MAY add dependencies (such as `editables`_) +that are necessary for their editable mechanism to function at runtime. + +The filename for the "editable" wheel MUST follow the wheel +:ref:`wheel-file-name-convention`; +it MAY use different :ref:`platform-compatibility-tags` +than for :ref:`build-interface-build-wheel`, +but its tags MUST be compatible with the platform this hook is executed on. + +If the :term:`Build Frontend` has previously called +:ref:`build-interface-prepare-metadata-for-build-editable` +and depends on the wheel resulting from this call +to have metadata matching this earlier call, +then it SHOULD provide the path to the created ``.dist-info`` directory +as the ``metadata_directory`` argument. +If this argument is provided, +then ``build_editable`` MUST produce a wheel with identical metadata. +The directory passed in by the build frontend MUST be identical +to the directory created by +:ref:`build-interface-prepare-metadata-for-build-editable`, +including any unrecognized files it created. + +An "editable" wheel uses the wheel format not for distribution +but as ephemeral communication between the build system and the front end. +This wheel MUST NOT be exposed to end users, nor cached, nor distributed. + + +.. _build-interface-get-requires-for-build-editable: + +get_requires_for_build_editable +------------------------------- + +.. code-block:: python + + def get_requires_for_build_editable(config_settings=None): + ... + +*Optional hook*. +This hook MUST return a list of strings containing +:ref:`dependency-specifiers`, +above and beyond those specified in the +:ref:`build-system.requires key ` +of :ref:`pyproject-toml-config-file`, +to be installed when calling the +:ref:`build-interface-build-editable` +or :ref:`build-interface-prepare-metadata-for-build-editable` hooks. + +If not defined by the :term:`Build Backend`, +the default implementation is equivalent to ``return []``. + + +.. _build-interface-prepare-metadata-for-build-editable: + +prepare_metadata_for_build_editable +----------------------------------- + +.. code-block:: python + + def prepare_metadata_for_build_editable(metadata_directory, config_settings=None): + ... + +*Optional hook*. +MUST create a :file:`.dist-info` directory containing :term:`Wheel` metadata +inside the specified ``metadata_directory``; +i.e., a directory like +:file:`{metadata_directory}/{package}-{version}.dist-info/`. +This MUST be a valid :file:`.dist-info` directory +as :ref:`defined in the wheel specification `, +except that it need not contain ``RECORD`` +or :ref:`signatures `. +The hook MAY also create other files inside this directory, +and a :term:`Build Frontend` MUST preserve, but otherwise ignore, such files; +the intention here is that in cases where +the metadata depends on build-time decisions, +the :term:`Build Backend` may need to record these decisions +in some convenient format for re-use by the actual wheel-building step. + +This MUST return the basename (not the full path) +of the :file:`.dist-info` directory it creates as a string. + +If a build frontend needs this information and the method is not defined, +it SHOULD call :ref:`build-interface-build-editable` +and look at the resulting metadata directly. + + +.. _build-interface-hook-invocation: + +Hook invocation +=============== + +The hooks MAY be called with positional or keyword arguments, +so backends implementing them SHOULD be careful to make sure that +their signatures match both the order and the names of the arguments above. + +All hooks MUST be run with the working directory set to the +root of the :term:`Source Tree` +(or :ref:`unpacked sdist `), +and MAY print arbitrary informational text to ``stdout`` and ``stderr``. +They MUST NOT read from ``stdin``, +and the build frontend MAY close ``stdin`` before invoking the hooks. + +The build frontend MAY capture ``stdout`` and/or ``stderr`` from the backend. +If the backend detects that an output stream is not a terminal/console +(e.g. ``not sys.stdout.isatty()``), +it SHOULD ensure that any output it writes to that stream is UTF-8 encoded. +The build frontend MUST NOT fail if captured output is not valid UTF-8, +but it MAY not preserve all the information in that case +(e.g. it may decode output using the ``'replace'`` error handler in Python). +If the output stream is a terminal, +the build backend is responsible for presenting its output accurately, +as for any program running in a terminal. + +If a hook raises an exception, or causes the process to terminate, +then this indicates an error. + + +.. _build-interface-config-settings: + +Config settings +=============== + +The ``config_settings`` argument, which is passed to all hooks, +is an arbitrary dictionary provided as an "escape hatch" +for users to pass ad-hoc configuration into individual package builds. +:term:`Build Backend`\s MAY assign any semantics they like to this dictionary. + +:term:`Build Frontend`\s SHOULD provide some mechanism for users to specify +arbitrary string-key/string-value pairs to be placed in this dictionary. +For example, they might support some syntax like ``--package-config CC=gcc``. +Build frontends MAY also provide arbitrary other mechanisms +for users to place entries in this dictionary. +For example, ``pip`` might choose to map the following mix +of modern and legacy command line arguments: + +.. code-block:: shell + + pip install \ + --package-config CC=gcc \ + --global-option="--some-global-option" \ + --build-option="--build-option1" \ + --build-option="--build-option2" + +into a ``config_settings`` dictionary as: + +.. code-block:: python + + { + "CC": "gcc", + "--global-option": ["--some-global-option"], + "--build-option": ["--build-option1", "--build-option2"], + } + +Of course, it is up to users to ensure that they pass options +which make sense for the particular build backend +and package that they are building. + + +.. _build-interface-build-environment: + +Build environment +================= + +One of the responsibilities of a :term:`Build Frontend` is +to set up the Python environment in which the :term:`Build Backend` will run. + +A build frontend MAY use any "virtual environment" mechanism it chooses; +such as virtualenv, venv, or no special mechanism at all. +However, whatever mechanism is used MUST meet the following criteria: + +- All dependencies required by the build backend + MUST be available for import from Python. + In particular: + + - The :ref:`build-interface-get-requires-for-build-wheel`, + :ref:`build-interface-get-requires-for-build-sdist` + and :ref:`build-interface-get-requires-for-build-editable` hooks + MUST be executed in an environment which contains the requires specified in + :ref:`build-system.requires in pyproject.toml `. + + - The :ref:`build-interface-prepare-metadata-for-build-wheel` + and :ref:`build-interface-build-wheel` hooks + MUST be executed in an environment which contains the + ``build-system.requires`` requirements + and those specified by the + :ref:`build-interface-get-requires-for-build-wheel` hook. + + - The :ref:`build-interface-build-sdist` hook + MUST be executed in an environment which contains the + ``build-system.requires`` requirements + and those specified by the + :ref:`build-interface-get-requires-for-build-sdist` hook. + + - The :ref:`build-interface-prepare-metadata-for-build-editable` + and :ref:`build-interface-build-editable` hooks + MUST be executed in an environment which contains the + ``build-system.requires`` requirements + and those specified by the + :ref:`build-interface-get-requires-for-build-editable` hook. + +- This MUST remain true even for new Python subprocesses + spawned by the build environment. + For example, code like: + + .. code-block:: python + + import subprocess, sys + subprocess.run([sys.executable, ...]) + + MUST spawn a Python process which has access to all the project's + build requirements. + This is necessary for build backends that want to + e.g. run legacy :file:`setup.py` scripts in a subprocess. + +- All command-line scripts provided by the build requirements + MUST be present in the build environment's ``PATH``. + For example, if a project declares a build-requirement on :ref:`flit`, + then the following MUST work + as a mechanism for running the Flit command-line tool: + + .. code-block:: python + + import shutil, subprocess + subprocess.run([shutil.which("flit"), ...]) + +A build backend MUST be prepared to function in any environment +which meets the above criteria. +In particular, it MUST NOT assume that it has access to any packages +except those that are present in the Python standard library, +or that are explicitly declared as build requirements. + +Frontends SHOULD call each hook in a fresh subprocess, +so that backends are free to change process global state +(such as environment variables or the working directory). +A Python library will be provided which frontends can use +to easily call hooks this way. + +Frontends MAY use any mechanism +for setting up a build environment that meets the above criteria, +including simply installing all build requirements into the global environment. +However, a build frontend SHOULD, by default, +create an isolated environment for each build, +containing only the Python standard library +and any explicitly requested build dependencies. + +Build frontends SHOULD provide some mechanism for users to override +the above defaults. +For example, a build frontend could have a +``--build-with-system-site-packages`` option that causes the +``--system-site-packages`` option to be passed to +virtualenv-or-equivalent when creating build environments, +or a ``--build-requirements-override=my-requirements.txt`` option that +overrides the project's normal build-time requirements. + + +.. _build-interface-frontend-requirements-for-editable-installs: + +Frontend requirements for editable installs +=========================================== + +:term:`Build Frontend`\s MUST install "editable" wheels +built with the :ref:`build-interface-build-editable` hook +in the same way as normal :term:`Wheel`\s +built with the :ref:`build-interface-build-wheel` hook. +This also means uninstallation of :term:`Editable Installation`\s +MUST NOT require any special treatment. + +Frontends MUST create a :file:`direct_url.json` file +in the :file:`.dist-info` directory of the installed distribution, +as specified in the :ref:`direct-url` specification. +The ``url`` value MUST be a ``file://`` URI to the :term:`Project` directory +(i.e. the directory containing the project's :term:`pyproject.toml`), +and the ``dir_info`` value MUST be ``{'editable': true}``. + +Frontends MUST execute :ref:`build-interface-get-requires-for-build-editable` +hooks in an environment which contains the +:ref:`build system requirements ` +specified in :ref:`pyproject-toml-config-file`. + +Frontends MUST execute the +:ref:`build-interface-prepare-metadata-for-build-editable` +and :ref:`build-interface-build-editable` hooks +in an environment which contains +the build system requirements from :file:`pyproject.toml` +and those specified by the +:ref:`build-interface-get-requires-for-build-editable` hook. + +Frontends MUST NOT expose the wheel obtained from +:ref:`build-interface-build-editable` to end users. +The wheel MUST be discarded after installation +and MUST NOT be cached nor distributed. + + +.. _`editables`: https://pypi.org/project/editables/ diff --git a/source/specifications/declaring-build-dependencies.rst b/source/specifications/declaring-build-dependencies.rst index 74f9c3ca2..c6d75808a 100644 --- a/source/specifications/declaring-build-dependencies.rst +++ b/source/specifications/declaring-build-dependencies.rst @@ -1,10 +1,252 @@ +.. _declaring-build-system: +================================== +Declaring a project's build system +================================== + +As originally specified in +:pep:`PEP 517 <517#source-trees>` and :pep:`PEP 518 <518#build-system-table>`, +projects may define a :ref:`build-system-table` +in their :ref:`pyproject.toml config file ` +to declare their :term:`Build Backend` +(as specified in the :ref:`build-interface` specification) +and its dependencies. + + +.. _build-system: +.. _build-system-table: + +``[build-system]`` table +======================== + +The ``[build-system]`` table is used to store build-related configuration. + +.. _build-system-table-default: + +Tools SHOULD NOT require the existence of the ``[build-system]`` table. +If it or a :term:`pyproject.toml` file is not present, +tools SHOULD assume the following default value: + +.. code-block:: toml + + [build-system] + # Minimum requirements for the build system to execute. + requires = ["setuptools"] + # The build backend Python object to invoke. + build-system = "setuptools.build_meta:__legacy__" + +If the table is present but is missing a value +for the mandatory :ref:`build-system-requires-key`, +tools SHOULD consider it an error. + +The valid top-level keys are listed below. +Keys not defined in this specification MUST NOT be added to this table. + + +.. _build-system-requires: +.. _build-system-requires-key: .. _declaring-build-dependencies: -=================================== -Declaring build system dependencies -=================================== +``requires`` key +---------------- + +The ``requires`` key is used to declare the Python-level dependencies +that must be installed in order to run +the project's :term:`Build Backend` successfully. + +The key's value MUST be a list of valid string :ref:`dependency-specifiers` +required to execute the +:ref:`specified build backend `. + +For a build tool such as :ref:`flit` with a backend package ``flit_core``, +an example ``requires`` value specifying a particular version range might be: + +.. code-block:: toml + + [build-system] + requires = ["flit_core >=3.2,<4"] + +Projects still relying on a legacy implicit :term:`setup.py` invocation +can specify the following value for the ``requires`` key: + +.. code-block:: toml + + [build-system] + requires = ["setuptools"] + +This is the :ref:`default value ` for this key +if the :ref:`build-system-table` is not present in a :term:`pyproject.toml`. +If the table is defined but is missing a value for the ``requires`` key, +tools SHOULD consider it an error. + +The following requirements also apply: + +- Project build requirements will define a directed graph of requirements + (project ``A`` needs ``B`` to build, ``B`` needs ``C`` and ``D``, etc.). + This graph MUST NOT contain cycles. + If (due to lack of co-ordination between projects, for example) + a cycle is present, :term:`Build Frontend`\s MAY refuse to build the project. +- Where build requirements are available as :term:`Wheel`\s, + frontends SHOULD use these where practical, to avoid deeply nested builds. + However, frontends MAY have modes + where they do not consider wheels when locating build requirements, + and so projects MUST NOT assume that publishing wheels + is sufficient to break a requirement cycle. +- Frontends SHOULD check explicitly for requirement cycles, + and SHOULD terminate the build with an informative message if one is found. + +.. note:: + + The requirement for no requirement cycles means that + backends wishing to self-host + (i.e., building a wheel for a backend uses that backend for the build) + need to make special provision to avoid causing cycles. + Typically, this will involve specifying themselves as an + :ref:`in-tree backend `, + and avoiding external build dependencies (usually by vendoring them). + + +.. _build-system-build-backend: +.. _build-system-build-backend-key: + +``build-backend`` key +--------------------- + +The ``build-backend`` key specifies the project's :term:`Build Backend`. +Its value MUST be a string naming the Python object +that exposes attributes with callables for each of +the :ref:`build-interface` hooks supported by the backend. +This is formatted following the same :file:`{module}:{object}` syntax as +an :ref:`entry point `. + +For example, with the value: + +.. code-block:: toml + + [build-system] + build-backend = "flit_core.buildapi:main" + +then the ``module`` would be ``flit_core.buildapi`` +and the ``object`` would be ``main``, +so the ``backend`` would be looked up by executing the equivalent of: + +.. code-block:: python + + import flit_core.buildapi + backend = flit_core.buildapi.main + +The ``object`` part MAY be omitted, +for cases where the importable ``module`` is the top-level ``backend`` object. +For example, with the value: + +.. code-block:: toml + + [build-system] + build-backend = "flit_core.buildapi" + +then the ``module`` would still be ``flit_core.buildapi`` +and the ``object`` part not specified, +so the ``backend`` would be looked up by executing the equivalent of: + +.. code-block:: python + + import flit_core.buildapi + backend = flit_core.buildapi + +Formally, the string SHOULD satisfy the grammar: + +.. code-block:: text + + identifier = (letter | '_') (letter | '_' | digit)* + module_path = identifier ('.' identifier)* + object_path = identifier ('.' identifier)* + entry_point = module_path (':' object_path)? + +which would import ``module_path`` +and then look up ``module_path.object_path`` +(or just ``module_path``, if no ``object_path`` is specified). + +When importing the module, +the directory containing the :term:`Source Tree` +MUST NOT be added to :data:`python:sys.path` and searched for the module, +including by Python's automatic behavior of adding +the working directory or script directory to the path, +unless present anyway due to :mod:`python:site` or :envvar:`python:PYTHONPATH`. + +If a ``build-backend`` key is not present within +a :ref:`build-system-table` of a :term:`pyproject.toml` file, +:term:`Build Frontend`\s SHOULD assume a default value for it of: + +.. code-block:: toml + + [build-system] + build-backend = "setuptools.build_meta:__legacy__" + +or else MAY revert to the legacy behaviour of directly executing +a :term:`setup.py` script at the root of the project's source tree. +Projects MAY still include a :file:`setup.py` +for compatibility with legacy tools that do not conform to this specification. + + +.. _build-system-backend-path: +.. _build-system-backend-path-key: +.. _build-system-in-tree-backend: + +``backend-path`` key +-------------------- + +The optional ``backend-path`` key specifies where +a local :term:`Build Backend` can be loaded from, +for projects that may wish to include the source code for their build backend +directly in their :term:`Source Tree`, +rather than referencing the backend via the :ref:`build-system-requires-key`. +Its value is a list of string paths to the directories which should be +inserted at the beginning of :data:`python:sys.path` to import the ``module`` +specified in the :ref:`build-system-build-backend-key`. + +For example, suppose a project has a backend ``object`` named ``backend_object`` +located inside a Python module located at +:file:`project_subdirectory/backend_directory/backend_package/backend_module.py` +relative to the project source tree root directory +(i.e. the directory in which the :file:`pyproject.toml` is located), +and with ``backend_package`` being a Python :term:`Import Package` +(i.e. with a :file:`__init__.py` file inside it). +Therefore, the ``build-backend`` and ``backend-path`` configuration would be: + +.. code-block:: toml + + [build-system] + build-backend = "backend_package.backend_module:backend_object" + backend-path = ["project_subdirectory/backend_directory"] + +Accordingly, ``project_subdirectory/backend_directory`` would be +inserted at the beginning of ``sys.path`` +and ``backend_package.backend_module`` would be imported from there, +with its ``backend_object`` attribute looked up as the ``backend`` object. +This is roughly equivalent to: + +.. code-block:: python + + import sys + sys.path.insert(0, "project_subdirectory/backend_directory") + import backend_package.backend_module + backend = backend_package.backend_module.backend_object + +There are restrictions on the content of the ``backend-path`` key: -`pyproject.toml` is a build system independent file format defined in :pep:`518` -that projects may provide in order to declare any Python level dependencies that -must be installed in order to run the project's build system successfully. +- Directories in ``backend-path`` are interpreted as + relative to the project root (i.e. the :file:`pyproject.toml` directory), + and MUST refer to a location within the :term:`Source Tree` + (after relative paths and symbolic links have been resolved). + :term:`Build Frontend`\s SHOULD check this condition + (typically by resolving the location to an absolute path + and resolving symbolic links, + and then checking that it is within the project root) + and fail with an error message if it is violated. +- The backend code MUST be loaded from one of + the directories specified in ``backend-path`` + (i.e., ``backend-path`` MUST NOT be specified without in-tree backend code). + Frontends MAY enforce this check, but are not required to. + Doing so would typically involve + checking the backend's :attr:`python:__file__` attribute + against the locations in ``backend-path``. diff --git a/source/specifications/declaring-project-metadata.rst b/source/specifications/declaring-project-metadata.rst index 51f46bd94..f2e730ffa 100644 --- a/source/specifications/declaring-project-metadata.rst +++ b/source/specifications/declaring-project-metadata.rst @@ -5,7 +5,7 @@ Declaring project metadata ========================== :pep:`621` specifies how to write a project's -:ref:`core metadata ` in a ``pyproject.toml`` file for +:ref:`core metadata ` in a :term:`pyproject.toml` file for packaging-related tools to consume. It defines the following specification as the canonical source for the format used. @@ -18,9 +18,9 @@ represents metadata that a tool will later provide. The fields defined in this specification MUST be in a table named ``[project]`` in ``pyproject.toml``. No tools may add fields to this table which are not defined by this specification. For tools wishing -to store their own settings in ``pyproject.toml``, they may use the -``[tool]`` table as defined in the -:ref:`build dependency declaration specification `. +to store their own settings in ``pyproject.toml``, +they may use the ``[tool]`` table +as :ref:`defined in the pyproject.toml specification `. The lack of a ``[project]`` table implicitly means the build back-end will dynamically provide all fields. @@ -83,7 +83,7 @@ The summary description of the project. The full description of the project (i.e. the README). The field accepts either a string or a table. If it is a string then -it is a path relative to ``pyproject.toml`` to a text file containing +it is a path relative to :term:`pyproject.toml` to a text file containing the full description. Tools MUST assume the file's encoding is UTF-8. If the file path ends in a case-insensitive ``.md`` suffix, then tools MUST assume the content-type is ``text/markdown``. If the file path @@ -94,7 +94,7 @@ specifying this field as ``dynamic``. For all unrecognized suffixes when a content-type is not provided, tools MUST raise an error. The ``readme`` field may also take a table. The ``file`` key has a -string value representing a path relative to ``pyproject.toml`` to a +string value representing a path relative to :term:`pyproject.toml` to a file containing the full description. The ``text`` key has a string value which is the full description. These keys are mutually-exclusive, thus tools MUST raise an error if the metadata @@ -129,7 +129,7 @@ The Python version requirements of the project. :ref:`License ` The table may have one of two keys. The ``file`` key has a string -value that is a file path relative to ``pyproject.toml`` to the file +value that is a file path relative to :term:`pyproject.toml` to the file which contains the license for the project. Tools MUST assume the file's encoding is UTF-8. The ``text`` key has a string value which is the license of the project. These keys are mutually exclusive, so a diff --git a/source/specifications/direct-url.rst b/source/specifications/direct-url.rst index 9b8180c47..9b30c4e74 100644 --- a/source/specifications/direct-url.rst +++ b/source/specifications/direct-url.rst @@ -78,7 +78,8 @@ When ``url`` refers to a local directory, the ``dir_info`` key MUST be present as a dictionary with the following key: - ``editable`` (type: ``boolean``): ``true`` if the distribution was installed - in editable mode, ``false`` otherwise. If absent, default to ``false``. + in :term:`Editable Mode`, ``false`` otherwise. + If absent, default to ``false``. When ``url`` refers to a local directory, it MUST have the ``file`` sheme and be compliant with :rfc:`8089`. In @@ -94,7 +95,8 @@ preserved when making relative paths absolute. A top-level ``subdirectory`` field MAY be present containing a directory path, relative to the root of the VCS repository, source archive or local directory, -to specify where ``pyproject.toml`` or ``setup.py`` is located. +to specify where :term:`pyproject.toml`, :term:`setup.cfg` or :term:`setup.py` +is located. .. note:: @@ -260,7 +262,7 @@ Local directory: "dir_info": {} } -Local directory installed in editable mode: +Local directory installed in :term:`Editable Mode`: .. code:: diff --git a/source/specifications/index.rst b/source/specifications/index.rst index 6a282b243..453c43f76 100644 --- a/source/specifications/index.rst +++ b/source/specifications/index.rst @@ -16,24 +16,26 @@ Package Distribution Metadata :maxdepth: 1 core-metadata + declaring-project-metadata version-specifiers dependency-specifiers - declaring-build-dependencies - declaring-project-metadata platform-compatibility-tags - recording-installed-packages entry-points direct-url -Package Distribution File Formats ---------------------------------- +Building and Installing +----------------------- .. toctree:: :maxdepth: 1 + pyproject-toml-config-file + declaring-build-dependencies + build-interface source-distribution-format binary-distribution-format + recording-installed-packages Package Index Interfaces diff --git a/source/specifications/pyproject-toml-config-file.rst b/source/specifications/pyproject-toml-config-file.rst new file mode 100644 index 000000000..374c75887 --- /dev/null +++ b/source/specifications/pyproject-toml-config-file.rst @@ -0,0 +1,75 @@ +.. _pyproject-toml-config-file: + +===================================== +The pyproject.toml configuration file +===================================== + +:term:`pyproject.toml` is a build-system-independent, tool-agnostic +file in `TOML format `__. +for storing Python build, packaging and tool configuration, +originally defined in :pep:`518` +and extended to include the :ref:`pyproject-toml-project-table` in :pep:`621`. + +It is typically located at the root of the :term:`Source Tree` +for Python :term:`Project`\s, +and included in the top-level directory of all :term:`Source Distribution`\s. + +The valid top-level tables are listed below. +Tables and their respective keys not specified here are reserved for future use, +to be proposed in future PEPs, +and MUST NOT be defined or used for any purpose. +Rather, the :ref:`pyproject-toml-tool-table` SHOULD be used instead. + + +.. _pyproject-toml-build-system: +.. _pyproject-toml-build-system-table: + +``[build-system]`` table +======================== + +The ``[build-system]`` table is used to store build-related configuration. +If the table is present but has no non-empty values, +tools SHOULD consider it an error. + +The contents and interpretation of the ``[build-system]`` table +are defined in the :ref:`declaring-build-system` specification. +Keys not defined in that specification MUST NOT be added to this table. + + +.. _pyproject-toml-project-table: +.. _pyproject-toml-metadata: + +``[project]`` table +=================== + +The ``[project]`` table is the standardized place to declare a project's +:ref:`core metadata ` for tools to consume. +The lack of a ``[project]`` table implicitly means that +the :term:`Build Backend` will dynamically provide all core metadata fields. +If the table is present but has no non-empty values, +tools MUST consider it an error. + +The contents and interpretation of the ``[project]`` table +are defined in the :ref:`declaring-project-metadata` specification. +Keys not defined in that specification MUST NOT be added to this table. + + +.. _pyproject-toml-tool: +.. _pyproject-toml-tool-table: + +``[tool]`` table +================ + +The ``[tool]`` table is where tools related to Python projects, +not just build systems, may allow users to specify configuration data. +Tools MUST use a named sub-table within ``[tool]`` to do this, +which to avoid collisions, MUST have as its key a name the project owns on +the :term:`Python Package Index (PyPI)`. +For example, the :ref:`flit` tool would store its configuration under +``[tool.flit]``. + +The contents and interpretation of each tool sub-table +are defined by each respective tool and its documentation. + + +.. _toml: https://toml.io/ diff --git a/source/specifications/source-distribution-format.rst b/source/specifications/source-distribution-format.rst index 3d6546502..3d9d30ff9 100644 --- a/source/specifications/source-distribution-format.rst +++ b/source/specifications/source-distribution-format.rst @@ -6,9 +6,9 @@ Source distribution format ========================== The current standard format of source distribution format is identified by the -presence of a :file:`pyproject.toml` file in the distribution archive. The layout -of such a distribution was originally specified in :pep:`517` and is formally -documented here. +presence of a :term:`pyproject.toml` file in the distribution archive. +The layout of such a distribution was originally specified in :pep:`517` +and is formally documented here. There is also the legacy source distribution format, implicitly defined by the behaviour of ``distutils`` module in the standard library, when executing @@ -23,12 +23,13 @@ Source distributions are also known as *sdists* for short. Source trees ============ -A *source tree* is a collection of files and directories -- like a version -control system checkout -- which contains a :file:`pyproject.toml` file that -can be use to build a source distribution from the contained files and -directories. :pep:`517` and :pep:`518` specify what is required to meet the -definition of what :file:`pyproject.toml` must contain for something to be -deemed a source tree. +A :term:`source tree` is a collection of files and directories +(typically a version control system checkout) +which contains a :term:`pyproject.toml` file that can be used +to build a source distribution from the contained files and directories. +:ref:`pyproject-toml-config-file` specifies what is required to meet the +definition of what :file:`pyproject.toml` must contain +for something to be deemed a source tree. Source distribution file name ============================= @@ -48,8 +49,8 @@ Source distribution file format A ``.tar.gz`` source distribution (sdist) contains a single top-level directory called ``{name}-{version}`` (e.g. ``foo-1.0``), containing the source files of the package. The name and version MUST match the metadata stored in the file. -This directory must also contain a :file:`pyproject.toml` in the format defined in -:ref:`declaring-build-dependencies`, and a ``PKG-INFO`` file containing +This directory must also contain a :term:`pyproject.toml` in the format defined +in :ref:`pyproject-toml-config-file`, and a ``PKG-INFO`` file containing metadata in the format described in the :ref:`core-metadata` specification. The metadata MUST conform to at least version 2.2 of the metadata specification. diff --git a/source/tutorials/installing-packages.rst b/source/tutorials/installing-packages.rst index c68ea6c18..d3fa6a8fa 100644 --- a/source/tutorials/installing-packages.rst +++ b/source/tutorials/installing-packages.rst @@ -377,16 +377,15 @@ In this case, this means to install any version "==1.4.*" version that's also Source Distributions vs Wheels ============================== -:ref:`pip` can install from either :term:`Source Distributions (sdist) ` or :term:`Wheels `, but if both are present -on PyPI, pip will prefer a compatible :term:`wheel `. You can override -pip`s default behavior by e.g. using its :ref:`--no-binary +:ref:`pip` can install from either :term:`Source Distribution`\s (sdists) +or :term:`Wheel`\s, but if both are present on PyPI, +pip will prefer a compatible wheel. +You can override pip`s default behavior by e.g. using its :ref:`--no-binary ` option. -:term:`Wheels ` are a pre-built :term:`distribution ` format that provides faster installation compared to :term:`Source -Distributions (sdist) `, especially when a -project contains compiled extensions. +:term:`Wheel` is a :term:`Built Distribution` format +that provides faster installation compared to :term:`Sdist`\s, +especially when a project contains compiled extensions. If :ref:`pip` does not find a wheel to install, it will locally build a wheel and cache it for future installs, instead of rebuilding the source distribution diff --git a/source/tutorials/managing-dependencies.rst b/source/tutorials/managing-dependencies.rst index eaa32f7ef..50c43ef96 100644 --- a/source/tutorials/managing-dependencies.rst +++ b/source/tutorials/managing-dependencies.rst @@ -169,13 +169,13 @@ and techniques, listed in alphabetical order, to see if one of them is a better Python applications, but not limited to them. * `PDM `_ for a modern Python package management tool supporting :pep:`582` (replacing virtual environments with ``__pypackages__`` - directory for package installation) and relying on standards such as :pep:`517` and - :pep:`621`. + directory for package installation) and relying on standards such as + the :ref:`build-interface` and :ref:`declaring-project-metadata` specifications. * `pip-tools `_ for creating a lock file of all dependencies from a list of packages directly used in a project, and ensuring that only those dependencies are installed. * `Poetry `__ for a tool comparable in scope to Pipenv that focuses more directly on use cases where the project being managed is - structured as a distributable Python package with a valid ``pyproject.toml`` file. + structured as a distributable Python package with a valid :term:`pyproject.toml` file. By contrast, Pipenv explicitly avoids making the assumption that the application being worked on will support distribution as a ``pip``-installable Python package. diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index bfb774335..b9709d92b 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -109,9 +109,9 @@ Creating pyproject.toml .. TODO: Add an intro sentence about pyproject.toml, and a sub-heading for "Configuring build tools" -:file:`pyproject.toml` tells "frontend" build tools like :ref:`pip` and -:ref:`build` what "backend" tool to use to create -:term:`distribution packages ` for your project. +The :term:`pyproject.toml` file tells :term:`Build Frontend` tools +like :ref:`pip` and :ref:`build` what :term:`Build Backend` tool to use to +create :term:`Distribution Package`\s for your project. You can choose from a number of backends; this tutorial uses :ref:`Hatchling ` by default, but it will work identically with :ref:`setuptools`, :ref:`Flit `, :ref:`PDM `, and others that support the ``[project]`` @@ -159,12 +159,12 @@ Open :file:`pyproject.toml` and enter one of these ``[build-system]`` tables: build-backend = "pdm.pep517.api" -- ``requires`` is a list of packages that are needed to build your package. You - don't need to install them; build frontends like :ref:`pip` will install them - automatically in a temporary, isolated virtual environment for use during the - build process. -- ``build-backend`` is the name of the Python object that frontends will use to - perform the build. +- ``requires`` is a list of packages that are needed to build your package. + You don't need to install them; :term:`Build Frontend`\s like :ref:`pip` + will install them automatically in a temporary, isolated virtual environment + for use during the build process. +- ``build-backend`` is the name of the Python :term:`Build Backend` object + that frontends will use to perform the build. .. TODO: Add note to check the tools' documentation for the current snippet? @@ -206,8 +206,8 @@ following this tutorial. username** for this tutorial, as this ensures you won't try to upload a package with the same name as one which already exists. - ``version`` is the package version. See the :ref:`version specifier specification ` - for more details on versions. Some build backends allow it to be specified - another way, such as from a file or a git tag. + for more details on versions. Some :term:`Build Backend`\s + allow it to be specified another way, such as from a file or a git tag. - ``authors`` is used to identify the author of the package; you specify a name and an email for each author. You can also list ``maintainers`` in the same format. @@ -288,8 +288,9 @@ Including other files --------------------- The files listed above will be included automatically in your -:term:`source distribution `. If you want to -include additional files, see the documentation for your build backend. +:term:`Source Distribution`. +If you want to include additional files, +see the documentation for your :term:`Build Backend`. .. _generating archives: @@ -341,8 +342,8 @@ files in the :file:`dist` directory: example_package_YOUR_USERNAME_HERE-0.0.1.tar.gz -The ``tar.gz`` file is a :term:`source distribution ` -whereas the ``.whl`` file is a :term:`built distribution `. +The ``tar.gz`` file is a :term:`Source Distribution`, +whereas the ``.whl`` file is a :term:`Built Distribution`. Newer :ref:`pip` versions preferentially install built distributions, but will fall back to source distributions if needed. You should always upload a source distribution and provide built distributions for the platforms your project is @@ -516,6 +517,7 @@ some things you can do: * Consider packaging tools that provide a single command-line interface for project management and packaging, such as :ref:`hatch`, :ref:`flit`, :ref:`pdm`, and :ref:`poetry`. -* Read :pep:`517` and :pep:`518` for background and details on build tool configuration. +* Read :ref:`pyproject-toml-config-file` and its linked specifications + for background and details on project and build tool configuration. * Read about :doc:`/guides/packaging-binary-extensions`.