Switch to hatchling backend #4747

vidipsingh · 2025-01-11T16:09:53Z

Replace setuptools with hatchling as build backend
Add hatch-vcs for Git tag-based versioning
Unify package versioning under a single source

Description

This PR switches PyBaMM's build system from setuptools to hatchling and implements Git tag-based versioning using hatch-vcs. This change simplifies our build process and version management now that PyBaMM is a pure Python package.

Related to: #4742

Type of change

New feature (non-breaking change which adds functionality)
Optimization (back-end change that speeds up the code)
Bug fix (non-breaking change which fixes an issue)

Key checklist:

No style issues: $ pre-commit run (or $ nox -s pre-commit)
All tests pass: $ python -m pytest (or $ nox -s tests)
The documentation builds: $ python -m pytest --doctest-plus src (or $ nox -s doctests)

You can run integration tests, unit tests, and doctests together at once, using $ nox -s quick.

Further checks:

Code is commented, particularly in hard-to-understand areas
Tests added that prove fix is effective or that feature works

codecov · 2025-01-11T16:28:34Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.69%. Comparing base (d4e884d) to head (0b8db17).
Report is 1 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #4747   +/-   ##
========================================
  Coverage    98.69%   98.69%           
========================================
  Files          303      303           
  Lines        23256    23256           
========================================
  Hits         22953    22953           
  Misses         303      303

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

agriyakhetarpal

Hi @vidipsingh, thanks for taking this up! I have just a few suggestions. Please remove other references of setuptools from this file (lines 174–189) and across other files as well, and replace them with hatch/hatchling as necessary.

pyproject.toml

tests/unit/test_util.py

pyproject.toml

kratman

There seems to be a lot of changes here that are not necessary for just switching to hatch. It should only be a couple lines.

Switching to hatch also requires changing the release workflow, the update version script, and other tools. There is a bunch of other stuff that needs to change

Saransh-cpp

Agree with @kratman. Switching to hatchling should be one PR, and adding hatch-vcs should be another. @vidipsingh can you restrict this PR to just switching to hatchling and getting rid of setuptool configurations?

The hatch-vcs PR can deal with updating the release workflow. Switching to hatchling will not change the release workflow.

In any case, both these PRs should go in after 25.1

vidipsingh · 2025-01-12T09:01:00Z

Agree with @kratman. Switching to hatchling should be one PR, and adding hatch-vcs should be another. @vidipsingh can you restrict this PR to just switching to hatchling and getting rid of setuptool configurations?

The hatch-vcs PR can deal with updating the release workflow. Switching to hatchling will not change the release workflow.

In any case, both these PRs should go in after 25.1

@Saransh-cpp Okay, I will restrict this PR to just switching to hatchling and will remove the setuptools configurations.

I also wanted to ask where else I need to remove references to setuptools apart from pyproject.toml? I’ve noticed it is referred to in the following files:

CONTRIBUTING.md
noxfile.py
.github/workflows/publish_pypi.yml
docs/source/api/parameters/parameter_sets.rst

Since I’m new to the PyBaMM repo, any guidance or suggestions regarding these changes would be greatly appreciated :)

Saransh-cpp · 2025-01-12T13:19:39Z

Sounds right! Some of the occurrences might have to be preserved, but we can point that out while reviewing the PR.

agriyakhetarpal

Thanks, @vidipsingh, looks good to me now!

pyproject.toml

agriyakhetarpal · 2025-01-13T20:44:38Z

Could you please rename the PR title, and replace the "Fixes" keyword in the PR description with "Related to" or similar so that the linked issue doesn't get closed, as we are no longer adding hatch-vcs in this PR?

agriyakhetarpal · 2025-01-13T20:45:08Z

@all-contributors please add @vidipsingh for infra

allcontributors · 2025-01-13T20:45:18Z

@agriyakhetarpal

I've put up a pull request to add @vidipsingh! 🎉

Actually, wait, I'm dismissing my review because the references to setuptools across other files still remain: #4747 (comment)

kratman · 2025-01-13T20:50:17Z

pyproject.toml

+    "src/pybamm/**/*.csv",
+    "src/pybamm/**/*.py",
+    "src/pybamm/CITATIONS.bib",
+    "src/pybamm/plotting/mplstyle",


I don't see this file in the repo

See my comment below.

kratman · 2025-01-13T20:50:39Z

pyproject.toml

+include = [
+    "src/pybamm/**/*.txt",
+    "src/pybamm/**/*.md",
+    "src/pybamm/**/*.csv",
+    "src/pybamm/**/*.py",
+    "src/pybamm/CITATIONS.bib",
+    "src/pybamm/plotting/mplstyle",


The comment explaining why this is not automatically generated is left below when the code was moved.

It would be nice if we could switch to this being automatic though

Makes sense, it's automatic for setuptools if include_package_data is set to True. Could you check if a similar option is available for hatchling, @vidipsingh?

It is not automatic for setuptools (you need to specify files in MANIFEST.in, as we do right now), but it is automatic for hatch 😉

See my comment below.

pyproject.toml

Saransh-cpp · 2025-01-13T21:39:42Z

pyproject.toml

 # List of files to include as package data. These are mainly the parameter CSV files in
 # the input/parameters/ subdirectories. Other files such as the CITATIONS file, relevant
 # README.md files, and specific .txt files inside the pybamm/ directory are also included.
 # These are specified to be included in the SDist through MANIFEST.in.


Suggested change

# List of files to include as package data. These are mainly the parameter CSV files in

# the input/parameters/ subdirectories. Other files such as the CITATIONS file, relevant

# README.md files, and specific .txt files inside the pybamm/ directory are also included.

# These are specified to be included in the SDist through MANIFEST.in.

Saransh-cpp · 2025-01-13T21:57:29Z

pyproject.toml

+[tool.hatch.build]
+packages = ["src/pybamm"]
+include = [
+    "src/pybamm/**/*.txt",
+    "src/pybamm/**/*.md",
+    "src/pybamm/**/*.csv",
+    "src/pybamm/**/*.py",
+    "src/pybamm/CITATIONS.bib",
+    "src/pybamm/plotting/mplstyle",


include-package-data was used to include the files from MANIFEST.in in the wheel file.

hatchling includes everything not ignored by .gitignore by default. Therefore, we should just include the required files -

Suggested change

[tool.hatch.build]

packages = ["src/pybamm"]

include = [

"src/pybamm/**/*.txt",

"src/pybamm/**/*.md",

"src/pybamm/**/*.csv",

"src/pybamm/**/*.py",

"src/pybamm/CITATIONS.bib",

"src/pybamm/plotting/mplstyle",

[tool.hatch]

build.targets.sdist.include = [

"src/pybamm",

"CITATION.cff",

]

We should not bundle the CITATION.cff file with the wheel (as it is being done now). We should also get rid off MANIFEST.in.

@vidipsingh This change is still needed

kratman · 2025-01-21T01:03:55Z

@vidipsingh Can you resolve the conflicts here?

- Replace setuptools with hatchling as build backend - Add hatch-vcs for Git tag-based versioning - Unify package versioning under a single source

- Replace setuptools with hatchling as build backend - Remove setuptools configurations - Revert to version 24.11.2

- Replace setuptools with hatch as build backend - Remove setuptools configurations - Add automatic file handling in hatch config

- Replace setuptools with hatchling as build backend - Add hatch-vcs for Git tag-based versioning - Unify package versioning under a single source

- Replace setuptools with hatchling as build backend - Remove setuptools configurations - Revert to version 24.11.2

- Replace setuptools with hatch as build backend - Remove setuptools configurations - Add automatic file handling in hatch config

kratman · 2025-01-21T19:37:42Z

@vidipsingh As a general rule, it is best not to force push once the review has started. It makes it really hard to see the changes. We squash most PRs (except releases), so the merge commits do not matter

kratman

It looks like you might have reverted a bunch of stuff from the previous reviews

pyproject.toml

noxfile.py

…m/vidipsingh/PyBaMM into switch-to-hatchling-and-hatch-vcs

kratman

Thanks this is looking good now. The failing test is a random failure, it is unrelated

pyproject.toml

Changes were made, looks like all review comments were covered

kratman · 2025-01-22T20:27:27Z

@Saransh-cpp, @agriyakhetarpal I checked all the previous comments, and they seem to be covered. So this should be ready to go

agriyakhetarpal

There are a few other references to setuptools that should be removed as well:

https://github.com/vidipsingh/PyBaMM/blob/adc7e0c3670ad2bf9571ff4f02d8472dda874ed3/pyproject.toml#L155-L170
MANIFEST.in, which was used by setuptools to configure what goes in the sdist, is now irrelevant and can be deleted

Happy to approve after, thanks!

Saransh-cpp

Thanks, @vidipsingh! Looks good to me!

kratman

@agriyakhetarpal We just need your review now

vidipsingh · 2025-01-23T17:12:54Z

Thanks @Saransh-cpp, @kratman, and @agriyakhetarpal for reviewing and merging my PR! Much appreciated :)

agriyakhetarpal · 2025-01-23T17:16:50Z

Thanks for your work! Please feel free to work on adding support for hatch-vcs and surrounding changes to our release workflow in a new PR if you wish to.

vidipsingh · 2025-01-23T17:31:47Z

Thanks for your work! Please feel free to work on adding support for hatch-vcs and surrounding changes to our release workflow in a new PR if you wish to.

Sure! I'll work on the hatch-vcs support and create a new PR for it.

vidipsingh requested review from martinjrobins, Saransh-cpp, agriyakhetarpal, kratman and arjxn-py as code owners January 11, 2025 16:09

agriyakhetarpal requested changes Jan 11, 2025

View reviewed changes

kratman requested changes Jan 12, 2025

View reviewed changes

Saransh-cpp requested changes Jan 12, 2025

View reviewed changes

agriyakhetarpal added the skip release Do not merge until the next release label Jan 12, 2025

agriyakhetarpal previously approved these changes Jan 13, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

allcontributors bot mentioned this pull request Jan 13, 2025

docs: add vidipsingh as a contributor for infra #4750

Merged

kratman requested changes Jan 13, 2025

View reviewed changes

vidipsingh changed the title ~~Switch to hatchling backend with hatch-vcs versioning~~ Switch to hatchling backend Jan 13, 2025

Saransh-cpp requested changes Jan 13, 2025

View reviewed changes

kratman removed the skip release Do not merge until the next release label Jan 15, 2025

vidipsingh added 6 commits January 22, 2025 00:37

build: switch to hatchling backend with hatch-vcs

cba8576

- Replace setuptools with hatchling as build backend - Add hatch-vcs for Git tag-based versioning - Unify package versioning under a single source

switch to hatchling without VCS integration

a92c4c7

- Replace setuptools with hatchling as build backend - Remove setuptools configurations - Revert to version 24.11.2

switch to hatch with automatic file handling

98a8e86

- Replace setuptools with hatch as build backend - Remove setuptools configurations - Add automatic file handling in hatch config

build: switch to hatchling backend with hatch-vcs

f3e8030

- Replace setuptools with hatchling as build backend - Add hatch-vcs for Git tag-based versioning - Unify package versioning under a single source

switch to hatchling without VCS integration

385674f

- Replace setuptools with hatchling as build backend - Remove setuptools configurations - Revert to version 24.11.2

switch to hatch with automatic file handling

b3a5de5

- Replace setuptools with hatch as build backend - Remove setuptools configurations - Add automatic file handling in hatch config

vidipsingh force-pushed the switch-to-hatchling-and-hatch-vcs branch from 53cda20 to b3a5de5 Compare January 21, 2025 19:31

Merge branch 'develop' into switch-to-hatchling-and-hatch-vcs

407289b

kratman previously requested changes Jan 21, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

pyproject.toml Outdated Show resolved Hide resolved

noxfile.py Show resolved Hide resolved

vidipsingh and others added 4 commits January 22, 2025 18:50

revert changes to the original state

8adc0ea

Merge branch 'develop' into switch-to-hatchling-and-hatch-vcs

42649a6

revert to original state before conflicts

3e1ca81

Merge branch 'switch-to-hatchling-and-hatch-vcs' of https://github.co…

537d68d

…m/vidipsingh/PyBaMM into switch-to-hatchling-and-hatch-vcs

kratman reviewed Jan 22, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

kratman requested review from Saransh-cpp, kratman and agriyakhetarpal January 22, 2025 20:22

Update pyproject.toml

adc7e0c

agriyakhetarpal reviewed Jan 22, 2025

View reviewed changes

remove setuptools references and MANIFEST.in

0b8db17

Saransh-cpp approved these changes Jan 23, 2025

View reviewed changes

kratman approved these changes Jan 23, 2025

View reviewed changes

agriyakhetarpal approved these changes Jan 23, 2025

View reviewed changes

kratman enabled auto-merge (squash) January 23, 2025 17:03

kratman merged commit 1e5b0e7 into pybamm-team:develop Jan 23, 2025
26 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch to hatchling backend #4747

Switch to hatchling backend #4747

vidipsingh commented Jan 11, 2025 •

edited

Loading

codecov bot commented Jan 11, 2025 •

edited

Loading

agriyakhetarpal left a comment

kratman left a comment

Saransh-cpp left a comment

vidipsingh commented Jan 12, 2025

Saransh-cpp commented Jan 12, 2025 •

edited

Loading

agriyakhetarpal left a comment

agriyakhetarpal commented Jan 13, 2025

agriyakhetarpal commented Jan 13, 2025

allcontributors bot commented Jan 13, 2025

kratman Jan 13, 2025

Saransh-cpp Jan 13, 2025

kratman Jan 13, 2025

agriyakhetarpal Jan 13, 2025

Saransh-cpp Jan 13, 2025 •

edited

Loading

Saransh-cpp Jan 13, 2025

Saransh-cpp Jan 13, 2025

kratman Jan 22, 2025

kratman commented Jan 21, 2025

kratman commented Jan 21, 2025

kratman left a comment

kratman left a comment

kratman commented Jan 22, 2025

agriyakhetarpal left a comment

Saransh-cpp left a comment

kratman left a comment

vidipsingh commented Jan 23, 2025

agriyakhetarpal commented Jan 23, 2025

vidipsingh commented Jan 23, 2025

Switch to hatchling backend #4747

Switch to hatchling backend #4747

Conversation

vidipsingh commented Jan 11, 2025 • edited Loading

Description

Type of change

Key checklist:

Further checks:

codecov bot commented Jan 11, 2025 • edited Loading

Codecov Report

agriyakhetarpal left a comment

Choose a reason for hiding this comment

kratman left a comment

Choose a reason for hiding this comment

Saransh-cpp left a comment

Choose a reason for hiding this comment

vidipsingh commented Jan 12, 2025

Saransh-cpp commented Jan 12, 2025 • edited Loading

agriyakhetarpal left a comment

Choose a reason for hiding this comment

agriyakhetarpal commented Jan 13, 2025

agriyakhetarpal commented Jan 13, 2025

allcontributors bot commented Jan 13, 2025

kratman Jan 13, 2025

Choose a reason for hiding this comment

Saransh-cpp Jan 13, 2025

Choose a reason for hiding this comment

kratman Jan 13, 2025

Choose a reason for hiding this comment

agriyakhetarpal Jan 13, 2025

Choose a reason for hiding this comment

Saransh-cpp Jan 13, 2025 • edited Loading

Choose a reason for hiding this comment

Saransh-cpp Jan 13, 2025

Choose a reason for hiding this comment

Saransh-cpp Jan 13, 2025

Choose a reason for hiding this comment

kratman Jan 22, 2025

Choose a reason for hiding this comment

kratman commented Jan 21, 2025

kratman commented Jan 21, 2025

kratman left a comment

Choose a reason for hiding this comment

kratman left a comment

Choose a reason for hiding this comment

kratman commented Jan 22, 2025

agriyakhetarpal left a comment

Choose a reason for hiding this comment

Saransh-cpp left a comment

Choose a reason for hiding this comment

kratman left a comment

Choose a reason for hiding this comment

vidipsingh commented Jan 23, 2025

agriyakhetarpal commented Jan 23, 2025

vidipsingh commented Jan 23, 2025

vidipsingh commented Jan 11, 2025 •

edited

Loading

codecov bot commented Jan 11, 2025 •

edited

Loading

Saransh-cpp commented Jan 12, 2025 •

edited

Loading

Saransh-cpp Jan 13, 2025 •

edited

Loading