Skip to content

fix: the generated sdist file should contain everything needed to check and test the code, and to build the documentation as well #967

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

fix: the generated sdist file should contain everything needed to check and test the code, and to build the documentation as well #967

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
edd604f
fix: the generated sdist file should contain everything needed to che…
jenstroeger Jun 30, 2025
098bcd1
chore: add CHANGELOG and SECURITY files to the sdist
jenstroeger Jul 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 5 additions & 8 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,20 +119,17 @@ omit = [
]


# https://flit.pypa.io/en/latest/pyproject_toml.html#sdist-section
# https://flit.pypa.io/en/stable/pyproject_toml.html#sdist-section
# See also: https://github.com/pypa/flit/issues/565
# See also: https://github.com/pypa/flit/discussions/745
[tool.flit.sdist]
include = []
Copy link

@takluyver takluyver Jul 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads up, what happens here will currently depend on whether you run flit build directly, or make an sdist through a generic tool that calls flit as a backend, like python -m build. With flit build, it currently uses info from git to include all checked-in files, like tests and docs sources, similar to git archive, except for those listed in exclude. However, Flit as a backend to general tools starts from only the files you need for installation. From Flit 4.0 this will be the default mode for flit build as well, although you'll be able to specify --use-vcs to keep the old behaviour.

https://flit.pypa.io/en/stable/pyproject_toml.html#including-files-committed-in-git-hg

So you might want to explicitly list includes for things you do want in the sdist.

(I can get into how we ended up in this scenario if you want. But the short version is, I like git-archive style sdists, but I don't think the backend can assume the presence of git, and I don't want it to behave differently if git is present.)

Copy link
Owner Author

@jenstroeger jenstroeger Jul 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @takluyver! With that in mind, I think it would make sense to

  • document source distribution better in our README; and
  • add a make build-from-sdist goal to our Makefile which takes an sdist and
    • runs the tests; then
    • produces a wheel (using build or flit build); then
    • produces the documentation.

@behnazh what do you think?

Copy link
Collaborator

@behnazh behnazh Sep 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the primary goals of using make build-from-sdist would be to reliably reproduce build artifacts I guess. To validate that the artifact built from source matches the original, it would be ideal to ensure the resulting package is bitwise equivalent. Achieving this level of reproducibility would eliminate the need for additional testing, and significantly simplifies the validation process.

Copy link
Owner Author

@jenstroeger jenstroeger Sep 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To validate that the artifact built from source matches the original, […]

With “original” I assume you mean a previously built wheel from the same commit hash? Basically:

  1. Build a wheel from the repo at commit hash H, then
  2. Build the sdist from the repo at commit hash H, then build a wheel from that sdist using the build timestamp from 1. and both wheels must be the same?

Copy link
Collaborator

@behnazh behnazh Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With “original” I assume you mean a previously built wheel from the same commit hash? Basically:

1. Build a wheel from the repo at commit hash _H_, then

2. Build the sdist from the repo at commit hash _H_, then build a wheel from that sdist using the build timestamp from 1. and both wheels must be the same?

Yes, exactly.

exclude = [
".github/",
"docs/",
"tests/",
".flake8",
".vscode/",
".gitignore",
".pre-commit-config.yaml",
"CHANGELOG.md",
"Makefile",
"SECURITY.md",
".gitattributes",
"CODEOWNERS",
]


Expand Down
Loading