Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PEP 517, PEP 518 & PEP 660 to specs and glossary #1111

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion source/discussions/wheel-vs-egg.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Here's a breakdown of the important differences between :term:`Wheel` and :term:
* :term:`Wheel` archives do not include .pyc files. Therefore, when the
distribution only contains Python files (i.e. no compiled extensions), and is
compatible with Python 2 and 3, it's possible for a wheel to be "universal",
similar to an :term:`sdist <Source Distribution (or "sdist")>`.
similar to an :term:`Sdist`.

* :term:`Wheel` uses :pep:`PEP376-compliant <376>` ``.dist-info``
directories. Egg used ``.egg-info``.
Expand Down
203 changes: 149 additions & 54 deletions source/glossary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,46 @@ Glossary
extensions.


Build Frontend

A tool that users interact with directly to trigger a build of
a :term:`Project`, which in turn invokes the project's
:term:`Build Backend` in a suitable environment
to generate a :term:`Built Distribution` (i.e. a :term:`Wheel`),
from a :term:`Source Tree` or :term:`Source Distribution`.
Examples include :ref:`build`, as well as :ref:`pip`
(when running a command such as ``pip wheel some-directory/``).
Compare to :term:`Integration Frontend`.
For more details,
see the :ref:`build-frontend-backend-interface` specification.


Build Backend

A tool directly responsible for transforming a
:term:`Source Tree` or :term:`Source Distribution`
Comment on lines +31 to +32
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should remove sdist since backends just get a simple directory, which frontends may populate with sdist contents rather than using the source.

into a :term:`Built Distribution` (i.e. a :term:`Wheel`).
Typically invoked by a :term:`Build Frontend` rather than directly.
Examples include :ref:`flit`, :ref:`hatch` and :ref:`setuptools`.
For more details,
see the :ref:`build-frontend-backend-interface` specification.


Built Distribution

A :term:`Distribution <Distribution Package>` format containing files
and metadata that only need to be moved to the correct location on the
target system, to be installed. :term:`Wheel` is such a format, whereas
distutil's :term:`Source Distribution <Source Distribution (or
"sdist")>` is not, in that it requires a build step before it can be
installed. This format does not imply that Python files have to be
precompiled (:term:`Wheel` intentionally does not include compiled
Python files).
A :term:`Distribution` format containing files and metadata
that only need to be moved to the correct location
on the target system to be installed.
:term:`Wheel` is such a format, whereas a :term:`Sdist` is not,
in that it requires processing by the :term:`Project`'s
:term:`Build Backend` before it can be installed.
This format does not imply that Python files have to be precompiled
(:term:`Wheel` intentionally does not include compiled Python files).


Distribution Package
Distribution
Package
Comment on lines +53 to +54
Copy link
Member

@pradyunsg pradyunsg Jul 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't add these -- we explicitly don't use the short forms here, to avoid ambiguity across the board. This is also called out in the definition here.

If there's places where we're that using the short form to direct at this, I'd rather that we update those references to be more explicit.


A versioned archive file that contains Python :term:`packages <Import
Package>`, :term:`modules <Module>`, and other resource files that are
Expand All @@ -39,6 +66,27 @@ Glossary
language distribution), which are often referred to with the single term
"distribution".


Editable Installation
Editable Mode

An installation mode implying that the :term:`Source Tree` of the
:term:`project` being installed is available in a local directory,
in which users expect that changes to its *Python* code
become effective without the need of a new installation step.

When a project is installed in "editable mode",
users expect it to behave as identically as practical
to a non-editable installation
(though some minor differences might be visible).
In particular, the code must be importable by other code,
and metadata must be available by standard mechanisms
such as ``importlib.metadata``.

Formally specified in :pep:`660` and now defined in the
:ref:`build-frontend-backend-interface` specification.


Egg

A :term:`Built Distribution` format introduced by :ref:`setuptools`,
Expand Down Expand Up @@ -75,6 +123,22 @@ Glossary
is needed to prevent confusion with a :term:`Distribution Package` which
is also commonly called a "package".


Integration Frontend

A tool that users run directly
that takes a set of :term:`Requirement`\s,
such as from a :term:`Project`'s :ref:`core-metadata`,
a :term:`Requirements File` or specified manually,
and attempts to update a working environment to satisfy them.
This may require locating, building and installing
a combination of :term:`Built Distribution`\s
and :term:`Source Distribution`\s,
including acting as a :term:`Build Frontend` in the latter case.
In a command like ``pip install lxml==2.4.0``,
:ref:`pip` is acting as an integration frontend.


Module

The basic unit of code reusability in Python, existing in one of two
Expand All @@ -96,26 +160,30 @@ Glossary

Project

A library, framework, script, plugin, application, or collection of data
or other resources, or some combination thereof that is intended to be
packaged into a :term:`Distribution <Distribution Package>`.

Since most projects create :term:`Distributions <Distribution Package>`
using either :pep:`518` ``build-system``, :ref:`distutils` or
:ref:`setuptools`, another practical way to define projects currently
is something that contains a :term:`pyproject.toml`, :term:`setup.py`,
or :term:`setup.cfg` file at the root of the project source directory.

Python projects must have unique names, which are registered on
:term:`PyPI <Python Package Index (PyPI)>`. Each project will then
contain one or more :term:`Releases <Release>`, and each release may
comprise one or more :term:`distributions <Distribution Package>`.
A library, framework, script, plugin, application,
collection of data or other resources, or some combination thereof
that is intended to be packaged into a :term:`Distribution`.
Comment on lines +163 to +165
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think applications are necessarily packaged, often just installed in a virtual environment with locked deps.

Copy link
Member

@pradyunsg pradyunsg Jul 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've definitely seen deployment mechanisms where the application is being packaged like a regular Python package, and shipped with all its dependencies, as a bundle of files. This is how, for example, https://shiv.readthedocs.io/en/latest/ works and pyinstaller works.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For sure! My point was just not always


Since most projects create :term:`Distribution`\s
using a :term:`Build Backend` :ref:`declared <declaring-build-system>`
within a :ref:`pyproject.toml file <pyproject-toml-config-file>`,
(or else implicitly use :ref:`setuptools`),
another practical way to define a project
is something that contains a :term:`pyproject.toml`
(or :term:`setup.py`/:term:`setup.cfg`) file
at the root of the project :term:`Source Tree`.

Python projects must have unique :ref:`names <core-metadata-name>`,
which are registered on a :term:`Package Index`
such as :term:`PyPI <Python Package Index (PyPI)>`.
Each project will contain one or more :term:`Releases <Release>`,
and each release may comprise one or more :term:`Distribution`\s.

Note that there is a strong convention to name a project after the name
of the package that is imported to run that project. However, this
doesn't have to hold true. It's possible to install a distribution from
the project 'foo' and have it provide a package importable only as
'bar'.
of the package that is imported to use that project.
However, this doesn't have to hold true.
It's possible to install a distribution from the project ``foo``
and have it provide a package importable only as ``bar``.


Pure Module
Expand Down Expand Up @@ -149,10 +217,13 @@ Glossary
domain name, ``pypi.python.org``, in 2017. It is powered by
:ref:`warehouse`.


pyproject.toml

The tool-agnostic :term:`Project` specification file.
Defined in :pep:`518`.
The tool-agnostic :term:`Project` configuration file.
Originally introduced in :pep:`518` and now defined in the
:ref:`pyproject-toml-config-file` specification.


Release

Expand All @@ -167,28 +238,33 @@ Glossary

Requirement

A specification for a :term:`package <Distribution Package>` to be
installed. :ref:`pip`, the :term:`PYPA <Python Packaging Authority
(PyPA)>` recommended installer, allows various forms of specification
that can all be considered a "requirement". For more information, see the
:ref:`pip:pip install` reference.
A specification for a :term:`Package`
to be installed by an :term:`Integration Frontend`.
:ref:`pip`, the :term:`PyPA <Python Packaging Authority (PyPA)>`
recommended installer,
allows various forms of specification
that can all be considered a "requirement".
For more information, see the :ref:`pip:pip install` reference.


Requirement Specifier

A format used by :ref:`pip` to install packages from a :term:`Package
Index`. For an EBNF diagram of the format, see the
`pkg_resources.Requirement
<https://setuptools.readthedocs.io/en/latest/pkg_resources.html#requirement-objects>`_
entry in the :ref:`setuptools` docs. For example, "foo>=1.3" is a
requirement specifier, where "foo" is the project name, and the ">=1.3"
portion is the :term:`Version Specifier`
A syntax used to declare the name and version of a :term:`Package`
that an :term:`Integration Frontend` such as :ref:`pip`
should install from a :term:`Package Index`.
For example, ``foo>=1.3`` is a requirement specifier,
where ``foo`` is the :ref:`project name <core-metadata-name>`
and ``>=1.3`` is the :term:`Version Specifier`.
The format was initially specified in :pep:`508`,
and is now defined in the :ref:`dependency-specifiers` specification.


Requirements File

A file containing a list of :term:`Requirements <Requirement>` that can
be installed using :ref:`pip`. For more information, see the :ref:`pip`
docs on :ref:`pip:Requirements Files`.
A file containing a list of :term:`Requirement`\s that can
be installed using an :term:`Integration Frontend`, such as :ref:`pip`.
For more information,
see the :ref:`pip` docs on :ref:`pip:Requirements Files`.


setup.py
Expand All @@ -200,17 +276,32 @@ Glossary

Source Archive

An archive containing the raw source code for a :term:`Release`, prior
to creation of a :term:`Source Distribution <Source Distribution (or
"sdist")>` or :term:`Built Distribution`.
An archive containing the :term:`Source Tree` for a :term:`Release`,
prior to creation of a
:term:`Source Distribution` or :term:`Built Distribution`.


Source Distribution
Sdist

A :term:`Distribution` format
(generated using, e.g., ``python -m build --sdist``)
that provides metadata and the essential source files needed
by a :term:`Build Backend` to generate a :term:`Built Distribution`
for installation by an installer like :ref:`pip`.

Source Distribution (or "sdist")

A :term:`distribution <Distribution Package>` format (usually generated
using ``python setup.py sdist``) that provides metadata and the
essential source files needed for installing by a tool like :ref:`pip`,
or for generating a :term:`Built Distribution`.
Source Tree

A collection of files and directories (typically from a VCS checkout)
containing the raw source code of a :term:`project`
that is used for development.
Can be stored in a :term:`Source Archive`
and is used by a :term:`Build Backend` to generate a
:term:`Source Distribution`
and in turn a :term:`Built Distribution`,
as well as directly in an :term:`Editable Installation`.
Typically contains a :ref:`pyproject-toml-config-file` at its root.


System Package
Expand All @@ -235,11 +326,15 @@ Glossary
wide. For more information, see the section on :ref:`Creating and using
Virtual Environments`.


Wheel

A :term:`Built Distribution` format introduced by :pep:`427`,
which is intended to replace the :term:`Egg` format. Wheel is currently
supported by :ref:`pip`.
A :term:`Built Distribution` format, introduced by :pep:`427`
and now defined in the :ref:`binary-distribution-format` specification,
which replaces the legacy :term:`Egg` format.
Wheel is supported by :ref:`pip` and other installation tools,
and is the primary output of :term:`Build Backend`\s.


Working Set

Expand Down
10 changes: 5 additions & 5 deletions source/guides/distributing-packages-using-setuptools.rst
Original file line number Diff line number Diff line change
Expand Up @@ -670,7 +670,7 @@ Working in "development mode"
You can install a project in "editable"
or "develop" mode while you're working on it.
When installed as editable, a project can be
edited in-place without reinstallation:
edited in-place without reinstallation:
changes to Python source files in projects installed as editable will be reflected the next time an interpreter process is started.

To install a Python package in "editable"/"development" mode
Expand All @@ -683,10 +683,10 @@ Change directory to the root of the project directory and run:

The pip command-line flag ``-e`` is short for ``--editable``, and ``.`` refers
to the current working directory, so together, it means to install the current
directory (i.e. your project) in editable mode. This will also install any
dependencies declared with ``install_requires`` and any scripts declared with
``console_scripts``. Dependencies will be installed in the usual, non-editable
mode.
directory (i.e. your project) in :term:`Editable Mode`.
This will also install any dependencies declared with ``install_requires``
and any scripts declared with ``console_scripts``.
Dependencies will be installed in the usual, non-editable mode.

You may want to install some of your dependencies in editable
mode as well. For example, supposing your project requires "foo" and "bar", but
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,8 @@ install and activate Python 3.10.

And now we can build dists from source. In this example, we'll
use ``build`` package, assuming that your project has a
``pyproject.toml`` properly set up (see
:pep:`517`/:pep:`518`).
:term:`pyproject.toml` properly set up
(see :ref:`pyproject-toml-config-file` for full details).

.. tip::

Expand Down Expand Up @@ -134,7 +134,7 @@ Now, whenever you push a tagged commit to your Git repository remote
on GitHub, this workflow will publish it to PyPI.
And it'll publish any push to TestPyPI which is useful for
providing test builds to your alpha users as well as making
sure that your release pipeline remains healthy!
sure that your release pipeline remains healthy!


.. _API token: https://pypi.org/help/#apitoken
Expand Down
4 changes: 2 additions & 2 deletions source/guides/tool-recommendations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ Packaging tool recommendations

* Use :ref:`setuptools` to define projects. [5]_ [6]_

* Use :ref:`build` to create :term:`Source Distributions
<Source Distribution (or "sdist")>` and :term:`wheels <Wheel>`.
* Use :ref:`build` to create :term:`Source Distribution`\s
and :term:`Wheel`\s.

If you have binary extensions and want to distribute wheels for multiple
platforms, use :ref:`cibuildwheel` as part of your CI setup to build
Expand Down
6 changes: 3 additions & 3 deletions source/guides/using-manifest-in.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
Including files in source distributions with ``MANIFEST.in``
============================================================

When building a :term:`source distribution <Source Distribution (or "sdist")>`
for your package, by default only a minimal set of files are included. You may
When building a :term:`Source Distribution` for your package,
by default only a minimal set of files are included. You may
find yourself wanting to include extra files in the source distribution, such
as an authors/contributors file, a :file:`docs/` directory, or a directory of
data files used for testing purposes. There may even be extra files that you
Expand Down Expand Up @@ -72,7 +72,7 @@ Command Description
(Files must be given as paths relative to the root of the project)
:samp:`recursive-include {dir-pattern} {pat1} {pat2} ...` Add all files under directories matching ``dir-pattern`` that match any of the listed patterns
:samp:`recursive-exclude {dir-pattern} {pat1} {pat2} ...` Remove all files under directories matching ``dir-pattern`` that match any of the listed patterns
:samp:`global-include {pat1} {pat2} ...` Add all files anywhere in the source tree matching any of the listed patterns
:samp:`global-include {pat1} {pat2} ...` Add all files anywhere in the :term:`Source Tree` matching any of the listed patterns
:samp:`global-exclude {pat1} {pat2} ...` Remove all files anywhere in the source tree matching any of the listed patterns
:samp:`graft {dir-pattern}` Add all files under directories matching ``dir-pattern``
:samp:`prune {dir-pattern}` Remove all files under directories matching ``dir-pattern``
Expand Down
Loading