Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build(DRAFT): Migrate from hatch to uv #3723

Draft
wants to merge 27 commits into
base: main
Choose a base branch
from

Conversation

dangotbanned
Copy link
Member

@dangotbanned dangotbanned commented Dec 29, 2024

Will close #3577

Important

A good chunk of the work is done, and uv is working.
Currently supporting both hatch and uv, temporarily.

Description

This PR paves the way for us to switch over to uv, with a solution for the issue I raised in (#3577 (comment)).

Using uv

Install uv for the first time or run:

uv self update

Install python:

uv python install 3.12

Go to the root of the repo:

cd altair/

Initialize a new virtual env:

uv venv -p 3.12 --seed

Install/update dependencies:

uv sync --all-extras

Task Runner

Right now, we can run "tasks" like:

uv run tools/tasks.py test

These can be arbitrarily combined like:

uv run tools/tasks.py generate-schema-wrapper update-init-file

I've tried my best to leave as many options open for us as possible.
You can define "tasks" programatically:

from tools._tasks import Tasks, Commands

app = Tasks(runner="uv")

@app.task()
def my_task() -> Commands:
    yield 'python -c "print(\"doing some work!\")"'
    yield "ruff check ."
    yield "pytest"

# Other tasks can refer to any previously declared tasks
@app.task()
def my_other_task() -> Commands:
    yield "my_task"
    yield "ruff format"

The other interface is demonstrated in tasks.toml.

Generating tasks.toml itself is also expressed as a "task":

altair/tools/tasks.py

Lines 219 to 222 in aaca9bc

@app.task("export-tasks")
def export_tasks() -> Commands:
TASKS_TOML = "tasks.toml"
yield cmd("from tools.tasks import app", f"app.to_path({TASKS_TOML!r})")

Future

Depending on if (astral-sh/uv#5903) takes a while to resolve, we can easily swap out tools.tasks.py for something like:

Note

This was pretty fun to work on, but is really just intended as a stepping stone towards using another tool.


TODO

The remaining work AFAIK:

https://ss64.com/bash/mkdir.html
We would want to see an error if `doc/` didn't exist. The docs wouldn't build as all the content is in that directory.

Working on migrating to a cross-platform shell that doesn't require this option anyway https://www.nushell.sh/commands/docs/mkdir.html
Decouples from `bash`, simplifies (#3577 (comment))
Provides 2 interfaces:
- `tasks.py`: to define using decorators
- `tasks.toml`: an example of defining in a similar way to `hatch` scripts

#3577 (comment)
Steps to reproduce:
- `uv self update` or install `uv` for the [first time](https://docs.astral.sh/uv/getting-started/installation/)
- `uv python install 3.12`
- `cd $ALTAIR_REPO_ROOT`
- `uv venv -p 3.12 --seed`
- `uv sync --all-extras`
Bug revealed after changing the `pytest` task to only contain `"pytest"`
- Add docs
- Handle semicolon joining in one place
- align `mkdir_cmd` with `mkdir -p` (https://ss64.com/bash/mkdir.html)
`*_cmd` functions and these variables are unrelated

Planning to namespace the functions, using `cmd`
All of these tools are now accessible under a single namespace, with discoverable docs
You can now safely run:
```py
import tools
tools.tasks.app.run(["lint", "format"])
```
Terms like "clean", "build", "publish" have another meaning when talking about the distribution
https://docs.astral.sh/uv/concepts/projects/build/
- Extra prep for string escaping while in a subprocess
- Use `--force` in `git add` to override (https://github.com/vega/altair/blob/7c2d97ffe80749a7cd1b24703547e40f4918172d/doc/.gitignore#L1)
- Raise a more informative error when a subprocess fails
All previous functionality is possible without this now.
Did some tidying up of what is left using (https://github.com/vega/vega-datasets/blob/369b462f7505e4ef3454668793e001e3620861ff/taplo.toml)

## Remaining sections
- `build`, `metadata`, `version` tables are used by `hatchling`
  - (https://hatch.pypa.io/latest/why/#build-backend)
- The `doc` environment is retained **only** due to how `_HatchRunner` is implemented
  - Otherwise we can remove that
  - (6e792a1)
- The test matrix functionality isn't something I've tried to reproduce, so left that in
  - Unsure how often others use it, but we could swap that out for `nox`
    - https://github.com/narwhals-dev/narwhals/blob/024d5d2294d1dcdd2e2f043d59eb03a3238c9e87/noxfile.py#L13
Comment on lines +144 to +151
if no_commit:
print(f"Unused commit message:\n {commit_message!r}")
elif branch != "main":
# FIXME: Unsure how to reproduce the RELEASING.md steps from this PR
# https://github.com/vega/altair/blob/main/RELEASING.md
print(f"Unable to push from {branch!r}.\n" "Must be on 'main'.")
else:
add_commit_push_github(commit_message)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FIXME

@mattijn I was wondering if you might be able to help me with this?
Looking at altair-viz.github.io(contributors), it seems like you've run these steps the most.

All the steps up this sequence are working:

# add commit, and push to github
git add . --all
git commit -m "$MSG"
git push origin master

These are the equivalent (with --force (404bb26)):

CMD_ADD = "git add . --all --force"
CMD_COMMIT = "git commit -m"
CMD_PUSH = "git push origin master"

Which are run in:

add_commit_push_github()

def add_commit_push_github(msg: str, /) -> None:
os.chdir(DOC_REPO_DIR)
print("Pushing ...")
# NOTE: Ensures the message uses cross-platform escaping
cmd_commit = *CMD_COMMIT.split(" "), msg
commands = (CMD_ADD, cmd_commit, CMD_PUSH)
for command in commands:
run_check(command)

Problem

Delete L146-L149 and it produces an error that I'm unsure how to solve.
AFAIK, it didn't seem to be related to permissions on https://github.com/altair-viz/altair-viz.github.io - but that is a blocker I'm expecting


RELEASING.md

altair/RELEASING.md

Lines 37 to 54 in 48e976e

7. Merge release branch into main, make sure that all required checks pass
8. On main, build source & wheel distributions. If you work on a fork, replace `origin` with `upstream`:
git switch main
git pull origin main
hatch clean # clean old builds & distributions
hatch build # create a source distribution and universal wheel
9. publish to PyPI (Requires correct PyPI owner permissions):
hatch publish
10. build and publish docs (Requires write-access to altair-viz/altair-viz.github.io):
hatch run doc:publish-clean-build
11. On main, tag the release. If you work on a fork, replace `origin` with `upstream`:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Switch from hatch to uv for package and environment management
1 participant