PEP 721: Use the data_filter when extracting tarballs, if it's available.

[encukou]

Previous behaviour is used on Python without PEP-720 tarfile filters. (Note that the feature is now in security releases of all supported versions.)

A custom filter (which wraps data_filter) is used to retain pip-specific behaviour:

• Removing a common leading directory
• Setting the mode (Unix permissions)

Compared to the previous behaviour, if a file can't be unpacked, the unpacking operation will fail with InstallError, rather than skipping the individual file with a logger.warning. This means that "some corrupt tar files" now can't be unpacked.
IMO, this is the correct path forward, but maybe there should be a CLI option to use the old behaviour? Sadly I don't know enough about pip to add one and propagate the setting to untar_file.

The new code path avoids internal API (TarFile._extract_member).

Note that PEP 721 limits itself to sdists, this change affects unpacking any other tar file.

I hope I got the nox/mypy/ruff stuff right, guess CI will tell...

---

This PR indents a big block of code. To hide whitespace changes, use git diff -w or add ?w=1 to the GitHub URL.

Fixes: #12111

[uranusjr]

I feel it may be a good idea to pull the except and else blocks into separate functions for readability. It’s difficult for me to find where the except block ends now.

[henryiii]

Needs same fix as pypa/build#675 to avoid breakage.

[encukou]

(This is still on my TODO list, it just keeps getting pushed back by conferences.)

[encukou]

Sorry for the delay.
A bug in my initial implementation of tarfile filters means that specific CPython versions raise spurious exceptions. Filtering those out, the “new” way unfortunately grew to be about as many lines as the “old” way -- although it does have more comments and a big chunk can be cleanly removed when support for specific versions ends.

[uranusjr]

I wonder if this would break some existing code. Do we want to add a flag to allow users to disable data_filter temporarily? Or should we treat those as security issues and allow breakage?

[pradyunsg]

Do we want to add a flag to allow users to disable data_filter temporarily? Or should we treat those as security issues and allow breakage?

The latter, IMO.