Merge dev to main branch (#1168)

* feat(core): Use tomllib on Python 3.11 (#1072)

* docs: 📝 Fix typo in `pip install pdm` description (#1061)

* Use tomllib on Python 3.11

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* news

* use a compatibility module

* missed one import

Co-authored-by: t106362512 <[email protected]>
Co-authored-by: hauntsaninja <>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* feat(core): Replace halo, click, and termcolor with rich (#1091)

* feat(core): Use `unearth` as the backend to find and download packages (#1096)

* perf(resolver): Speed up the resolution with lazy find_matches (#1098)

* Improve the output of installing packages

* Use confirm instead of ask

* feat(core): New command: pdm publish (#1107)

* Use rich handler for logging to stderr

* feat(scripts): added composite tasks support (#1117)

* feat(core): Add option to skip hooks (#1127)

* fix(scripts): allow pdm test command to receive path arguments

* feat(hooks): added a `--skip` option to skipp scripts and hooks (#1127)

fix #948

* feat(hooks): use PDM_SKIP_HOOKS environement variable as fallback for skip list

* feat(core): Support setup.py import (#1137)

* Update completion script

* fix(resolution): fix a bug that versions with local part can't be found and installed
Close #1093

* feat(core): forbid editable depenencies in project table (#1140)

* Make the error message more friendly

* doc: improve the docs about dependencies

* doc: add CLI reference doc

* doc: use asciiart as the program description

* chore: remove remaining artifacts from #1127 (#1152)

* Feature: complete lifecycle signals and documentation (#1147)

* feat(hooks): Added pre-publish hook

* refactor(hooks): dynamic signal/hooks listing avoiding double declaration

* feat(hooks): added (pre|post)_script and (pre|post)_run hooks

* doc(hooks): added lifecycle and hooks documentation

* review fix

* fix(tests): add and use the _echo fixture for cross-plateform and concise test echos

* refactor(hooks): automatically register the script handler for all hooks

* feat: Update pdm-pep517 to 1.0 (#1153)

* fix(scripts): merge the Script and Description field from listing (#1151)

* feat: fetch the candidate hashes concurrently (#1154)

* feat: fetch the candidate hashes concurrently

* add news

* Feat/respect-source-order (#1155)

* doc: restructure the docs about project metadata and build configuration

* parse pep 621 metadata to avoid build (#1156)

* feat: Remove the compatible support for pdm legacy metadata (#1157)

* fix(config): use platform standard directories for all PDM directories (#1161)

Fixes #1150

* fix(#1156): only trust parsing result when all are static

* New build configuration table

* chore: added a tox.ini file for easier local testing against all Python versions (#1160)

* feat(CLI): Yarn-like root scripts fallback (#1159)

* feat(hooks): added a post_use hook (#1163)

Co-authored-by: Shantanu <[email protected]>
Co-authored-by: t106362512 <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Axel H <[email protected]>

.github/workflows/ci.yml

@@ -50,11 +50,6 @@ jobs:
     steps:
       - uses: actions/checkout@v3
 
-      - name: Set Python 2.7
-        uses: actions/setup-python@v3
-        with:
-          python-version: 2.7
-          architecture: ${{ matrix.arch }}
       - name: Set Python 3.6
         uses: actions/setup-python@v3
         with:
@@ -78,6 +73,12 @@ jobs:
         with:
           python-version: 3.9
           architecture: ${{ matrix.arch }}
+      - name: Set Python 3.10
+        uses: actions/setup-python@v2
+        if: matrix.python-version != '3.10'
+        with:
+          python-version: "3.10"
+          architecture: ${{ matrix.arch }}
       - name: Set up Python ${{ matrix.python-version }}
         uses: actions/setup-python@v3
         with:

.pre-commit-config.yaml

@@ -5,7 +5,6 @@ repos:
     rev: 22.3.0
     hooks:
       - id: black
-        exclude: ^pdm/_vendor
 
   - repo: https://github.com/PyCQA/flake8
     rev: 4.0.1
@@ -26,5 +25,4 @@ repos:
         args: [pdm]
         pass_filenames: false
         additional_dependencies:
-          - types-setuptools
-          - types-toml
+          - types-requests

CONTRIBUTING.md

@@ -75,6 +75,20 @@ pdm run test
 
 The test suite is still simple and needs expansion! Please help write more test cases.
 
+!!! note
+    You can also run your test suite against all supported Python version using `tox` with the `tox-pdm` plugin.
+    You can either run it by yourself with:
+
+    ```shell
+    tox
+    ```
+
+    or from `pdm` with:
+
+    ```shell
+    pdm run tox
+    ```
+
 ### Code style
 
 PDM uses `pre-commit` for linting. Install `pre-commit` first, then:

docs/docs/configuration.md

@@ -1,15 +1,15 @@
 ## Available Configurations
 
-The following configuration items can be retrieved and modified by `pdm config` command.
+The following configuration items can be retrieved and modified by [`pdm config`](usage/cli_reference.md#exec-0--config) command.
 
 | Config Item                   | Description                                                               | Default Value                                                             | Available in Project | Env var                  |
 | ----------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------- | -------------------- | ------------------------ |
 | `build_isolation`             | Isolate the build environment from the project environment                | Yes                                                                       | True                 | `PDM_BUILD_ISOLATION`    |
 | `cache_dir`                   | The root directory of cached files                                        | The default cache location on OS                                          | No                   |                          |
 | `check_update`                | Check if there is any newer version available                             | True                                                                      | No                   |                          |
 | `global_project.fallback`     | Use the global project implicitly if no local project is found            | `False`                                                                   | No                   |                          |
-| `global_project.path`         | The path to the global project                                            | `~/.pdm/global-project`                                                   | No                   |                          |
-| `global_project.user_site`    | Whether to install to user site                                            | `False`                                                                   | No                   |                          |
+| `global_project.path`         | The path to the global project                                            | `<default config location on OS>/global-project`                          | No                   |                          |
+| `global_project.user_site`    | Whether to install to user site                                           | `False`                                                                   | No                   |                          |
 | `install.cache`               | Enable caching of wheel installations                                     | False                                                                     | Yes                  |                          |
 | `install.cache_method`        | Specify how to create links to the caches(`symlink` or `pth`)             | `symlink`                                                                 | Yes                  |                          |
 | `install.parallel`            | Whether to perform installation and uninstallation in parallel            | `True`                                                                    | Yes                  | `PDM_PARALLEL_INSTALL`   |

docs/docs/pyproject/build.md

@@ -0,0 +1,249 @@
+# Build Configuration
+
+`pdm` uses the [PEP 517](https://www.python.org/dev/peps/pep-0517/) to build the package.
+A build backend is what drives the build system to build source distributions and wheels from arbitrary source trees.
+
+`pdm` also ships with its own build backend, [`pdm-pep517`](https://pypi.org/project/pdm-pep517/). Besides the [PEP 621 project meta](pep621.md), it also reads some additional configurations to control the build behavior. To use it, include the following in your `pyproject.toml`(It will be done automatically if you use the [`pdm init`](../usage/cli_reference.md#exec-0--init) or [`pdm import`](../usage/cli_reference.md#exec-0--import) to create the file):
+
+```toml
+[build-system]
+requires = ["pdm-pep517"]
+build-backend = "pdm.pep517.api"
+```
+
+## Dynamic versioning
+
+`pdm-pep517` supports dynamic versions from two sources. To enable dynamic versioning, remember to include `version` in the `dynamic` field of PEP 621 metadata:
+
+```toml
+[project]
+...
+dynamic = ["version"]
+```
+
+### Dynamic version from file
+
+```toml
+[tool.pdm]
+version = { source = "file", path = "mypackage/__version__.py" }
+```
+
+The backend will search for the pattern `__version__ = "{version}"` in the given file and use the value as the version.
+
+!!! TIP
+
+    Thanks to the TOML syntax, the above example is equivalent to the following:
+
+    ```toml
+    [tool.pdm.version]
+    source = "file"
+    path = "mypackage/__version__.py"
+    ```
+    Or:
+    ```toml
+    [tool.pdm]
+    version.source = "file"
+    version.path = "mypackage/__version__.py"
+    ```
+
+### Dynamic version from SCM
+
+If you've used [`setuptools-scm`](https://pypi.org/project/setuptools-scm/) you will be familiar with this approach. `pdm-pep517` can also read the version from the tag of your SCM repository:
+
+```toml
+[tool.pdm]
+version = { source = "scm" }
+```
+
+#### Specify the version manually
+
+When building the package, `pdm-pep517` will require the SCM to be available to populate the version. If that is not the case, you can still specify the version with an environment variable `PDM_PEP517_SCM_VERSION`:
+
+```bash
+export PDM_PEP517_SCM_VERSION="1.2.3"
+pdm build
+```
+
+#### Write the version to file
+
+For dynamic version read from SCM, it would be helpful to write the evaluated value to a file when building a wheel, so that you do not need `importlib.metadata` to get the version in code.
+
+```toml
+[tool.pdm.version]
+source = "scm"
+write-to = "mypackage/__version__.py"
+write-template = "__version__ = '{}'"  # optional, default to "{}"
+```
+
+For source distributions, the version will be *frozen* and converted to a static version in the `pyproject.toml` file, which will be included in the distribution.
+
+
+## Include and exclude files
+
+To include extra files and/or exclude files from the distribution, given the paths in `includes` and `excludes` configuration, as glob patterns:
+
+```toml
+[tool.pdm.build]
+includes = [
+    "**/*.json",
+    "mypackage/",
+]
+excludes = [
+    "mypackage/_temp/*"
+]
+```
+
+In case you may want some files to be included in sdist only, use the `source-includes` field:
+
+```toml
+[tool.pdm.build]
+includes = [...]
+excludes = [...]
+source-includes = ["tests/"]
+```
+
+Note that the files defined in `source-includes` will be **excluded** automatically from binary distributions.
+
+### Default values for includes and excludes
+
+If you don't specify any of these fields, PDM can determine the values for you to fit the most common workflows, in the following manners:
+
+- Top-level packages will be included.
+- `tests` package will be excluded from **non-sdist** builds.
+- `src` directory will be detected as the `package-dir` if it exists.
+
+If your project follows the above conventions you don't need to config any of these fields and it just works.
+Be aware PDM won't add [PEP 420 implicit namespace packages](https://www.python.org/dev/peps/pep-0420/) automatically and they should always be specified in `includes` explicitly.
+
+## Select another package directory to look for packages
+
+Similar to `setuptools`' `package_dir` setting, one can specify another package directory, such as `src`, in `pyproject.toml` easily:
+
+```toml
+[tool.pdm.build]
+package-dir = "src"
+```
+
+If no package directory is given, PDM can also recognize `src` as the `package-dir` implicitly if:
+
+1. `src/__init__.py` doesn't exist, meaning it is not a valid Python package, and
+2. There exist some packages under `src/*`.
+
+## Implicit namespace packages
+
+As specified in [PEP 420](https://www.python.org/dev/peps/pep-0420), a directory will be recognized as a namespace package if:
+
+1. `<package>/__init__.py` doesn't exist, and
+2. There exist normal packages and/or other namespace packages under `<package>/*`, and
+3. `<package>` is explicitly listed in `includes`
+
+## Custom file generation
+
+During the build, you may want to generate other files or download resources from the internet. You can achieve this by the `setup-script` build configuration:
+
+```toml
+[tool.pdm.build]
+setup-script = "build.py
+```
+
+In the `build.py` script, `pdm-pep517` looks for a `build` function and calls it with two arguments:
+
+- `src`: (str) the path to the source directory
+- `dst`: (str) the path to the distribution directory
+
+Example:
+
+```python
+# build.py
+def build(src, dst):
+    target_file = os.path.join(dst, "mypackage/myfile.txt")
+    os.makedirs(os.path.dirname(target_file), exist_ok=True)
+    download_file_to(dst)
+```
+
+The generated file will be copied to the resulted wheel with the same hierarchy, you need to create the parent directories if necessary.
+
+## Build Platform-specific Wheels
+
+`setup-script` can also be used to build platform-specific wheels, such as C extensions. Currently, building C extensions still relies on `setuptools`.
+
+Set the `run-setuptools` to `true` under `setup-script`, and `pdm-pep517` will generate a `setup.py` with the custom `build` function in the script and run `python setup.py build` to build extensions.
+
+In the `build.py` script, the expected `build` function receives the argument dictionary to be passed to the `setup()` call. In the function, you can update the dictionary with any additional or changed values as you want.
+
+Here is an example adapted to build `MarkupSafe`:
+
+```python
+# build.py
+from setuptools import Extension
+
+ext_modules = [
+    Extension("markupsafe._speedups", ["src/markupsafe/_speedups.c"])
+]
+
+def build(setup_kwargs):
+    setup_kwargs.update(ext_modules=ext_modules)
+```
+
+The build configuration should look like:
+
+```toml
+# pyproject.toml
+[tool.pdm.build]
+setup-script = "build.py"
+run-setuptools = true
+```
+
+If you run [`pdm build`](../usage/cli_reference.md#exec-0--build)(or any other build frontends such as [build](https://pypi.org/project/build)), PDM will build a platform-specific wheel file as well as a sdist.
+
+By default, every build is performed in a clean and isolated environment, only build requirements can be seen. If your build has optional requirements that depend on the project environment, you can turn off the environment isolation by `pdm build --no-isolation` or setting config `build_isolation` to falsey value.
+
+### Override the "Is-Purelib" value
+
+Sometimes you may want to build platform-specific wheels but don't have a build script(the binaries may be built or fetched by other tools). In this case
+you can set the `is-purelib` value in the `pyproject.toml` to `false`:
+
+```toml
+[tool.pdm.build]
+is-purelib = false
+```
+
+If this value is not specified, `pdm-pep517` will build platform-specific wheels if `run-setuptools` is `true`, therefore `is-purelib` is `true` in this case.
+
+## Editable build backend
+
+PDM implements [PEP 660](https://www.python.org/dev/peps/pep-0660/) to build wheels for editable installation.
+One can choose how to generate the wheel out of the two methods:
+
+- `path`: (Default)The legacy method used by setuptools that create .pth files under the packages path.
+- `editables`: Create proxy modules under the packages path. Since the proxy module is looked for at runtime, it may not work with some static analysis tools.
+
+Read the PEP for the difference of the two methods and how they work.
+
+Specify the method in pyproject.toml like below:
+
+```toml
+[tool.pdm.build]
+editable-backend = "path"
+```
+
+`editables` backend is more recommended but there is a known limitation that it can't work with PEP 420 namespace packages.
+So you would need to change to `path` in that case.
+
+!!! note "About Python 2 compatibility"
+    Due to the fact that the build backend for PDM managed projects requires Python>=3.6, you would not be able to
+    install the current project if Python 2 is being used as the host interpreter. You can still install other dependencies not PDM-backed.
+
+## Use other PEP 517 backends
+
+Apart from `pdm-pep517`, `pdm` plays well with any PEP 517 build backends that comply with PEP 621 specification. At the time of writing, [`flit`](https://pypi.org/project/flit)(backend: `flit-core`) and [`hatch`](https://pypi.org/project/hatch)(backend: `hatchling`) are working well with PEP 621 and [`setuptools`](https://pypi.org/project/setuptools) has experimental support. To use one of them, you can specify the backend in the `pyproject.toml`:
+
+```toml
+[build-system]
+requires = ["flit_core >=2,<4"]
+build-backend = "flit_core.buildapi"
+```
+
+PDM will call the correct backend when doing [`pdm build`](../usage/cli_reference.md#exec-0--build).
+
+