Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch workflows to use new nightly iree-turbine packages. #657

Merged
merged 5 commits into from
Dec 9, 2024
Merged

Switch workflows to use new nightly iree-turbine packages. #657

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a001373
Switch workflows to use new nightly iree-turbine packages.
ScottTodd Dec 6, 2024
e7a8255
Update packages in sharktank workflow (could also use venv + no cache).
ScottTodd Dec 6, 2024
99aec1f
Add `--upgrade`. Could also install _before_ sharktank/shortfin.
ScottTodd Dec 6, 2024
c2319c4
Switch `test_with_data` job to use venv.
ScottTodd Dec 7, 2024
53dbcfb
Merge branch 'main' into iree-turbine-nightly
ScottTodd Dec 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions .github/workflows/ci-llama-large-tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,12 @@ jobs:
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -r requirements.txt -r sharktank/requirements-tests.txt -e sharktank/

# Install latest iree-turbine.
pip install --no-compile -f https://iree.dev/pip-release-links.html --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"

# Test with nightly releases, not what iree-turbine uses.
pip install -f https://iree.dev/pip-release-links.html --upgrade --pre \
# Install nightly IREE packages.
# We could also pin to a known working or stable version.
pip install -f https://iree.dev/pip-release-links.html --pre --upgrade \
iree-base-compiler \
iree-base-runtime
iree-base-runtime \
iree-turbine
Copy link
Collaborator

@archana-ramalingam archana-ramalingam Dec 7, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QQ: Shouldn't installing iree-turbine get us the latest/relevant version of compiler and runtime. Do we have to explicitly install/update them too?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. Good question. It does.

pip install --find-links https://iree.dev/pip-release-links.html iree-turbine --upgrade --pre

Collecting iree-turbine
  Downloading https://github.com/iree-org/iree-turbine/releases/download/dev-wheels/iree_turbine-3.1.0rc20241206-py3-none-any.whl (292 kB)
     ---------------------------------------- 292.3/292.3 kB 18.8 MB/s eta 0:00:00
Collecting numpy
  Downloading numpy-2.2.0rc1-cp311-cp311-win_amd64.whl (12.9 MB)
     ---------------------------------------- 12.9/12.9 MB 81.8 MB/s eta 0:00:00
Collecting iree-base-compiler
  Downloading https://github.com/iree-org/iree/releases/download/iree-3.1.0rc20241206/iree_base_compiler-3.1.0rc20241206-cp311-cp311-win_amd64.whl (49.3 MB)
     ---------------------------------------- 49.3/49.3 MB 34.4 MB/s eta 0:00:00
Collecting iree-base-runtime
  Downloading https://github.com/iree-org/iree/releases/download/iree-3.1.0rc20241206/iree_base_runtime-3.1.0rc20241206-cp311-cp311-win_amd64.whl (5.3 MB)
     ---------------------------------------- 5.3/5.3 MB 15.5 MB/s eta 0:00:00

pip freeze

filelock==3.16.1
fsspec==2024.10.0
iree-base-compiler==3.1.0rc20241206
iree-base-runtime==3.1.0rc20241206
iree-turbine==3.1.0rc20241206
Jinja2==3.1.4
MarkupSafe==3.0.2
ml_dtypes==0.5.0
mpmath==1.3.0
networkx==3.4.2
numpy==2.2.0rc1
sympy==1.13.1
torch==2.5.1
typing_extensions==4.12.2

On one hand, we could do the minimal thing and install just iree-turbine and let it pull the other deps. On the other, we could be explicit, which I think I like more, given how we do directly use some of the underlying packages:
image


pip freeze

Expand Down
12 changes: 5 additions & 7 deletions .github/workflows/ci-llama-quick-tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,12 @@ jobs:
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -r requirements.txt -r sharktank/requirements-tests.txt -e sharktank/

# Install latest iree-turbine.
pip install --no-compile -f https://iree.dev/pip-release-links.html --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"

# Test with nightly releases, not what iree-turbine uses.
pip install -f https://iree.dev/pip-release-links.html --upgrade --pre \
# Install nightly IREE packages.
# We could also pin to a known working or stable version.
pip install -f https://iree.dev/pip-release-links.html --pre --upgrade \
iree-base-compiler \
iree-base-runtime
iree-base-runtime \
iree-turbine

pip freeze

Expand Down
9 changes: 3 additions & 6 deletions .github/workflows/ci-sglang-benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,11 @@ jobs:
# from non default locations first. Installing the PyTorch CPU
# wheels saves multiple minutes and a lot of bandwidth on runner setup.
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -f https://iree.dev/pip-release-links.html --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"
pip install -f https://iree.dev/pip-release-links.html --pre iree-turbine
pip install --no-compile -r requirements.txt -e sharktank/ shortfin/

# Try with the latest nightly releases, not what iree-turbine pins.
# We could also pin to a known working or stable version.
# This should eventually stabilize. Do the best we can for now.
pip install -f https://iree.dev/pip-release-links.html --upgrade \
# Pin to known-working versions.
pip install -f https://iree.dev/pip-release-links.html --pre --upgrade \
iree-base-compiler==3.1.0rc20241204 \
iree-base-runtime==3.1.0rc20241204 \
"numpy<2.0"
Expand Down
5 changes: 2 additions & 3 deletions .github/workflows/ci-sglang-integration-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,15 +54,14 @@ jobs:
# from non default locations first. Installing the PyTorch CPU
# wheels saves multiple minutes and a lot of bandwidth on runner setup.
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -f https://iree.dev/pip-release-links.html --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"
pip install --no-compile -r requirements.txt -e sharktank/ shortfin/

# Use newest possible releases to be able to track commits that may
# cause errors.
pip install -f https://iree.dev/pip-release-links.html --upgrade --pre \
pip install -f https://iree.dev/pip-release-links.html --pre --upgrade \
iree-base-compiler \
iree-base-runtime \
iree-turbine \
"numpy<2.0"

# Install SGLang and sentence_transformers
Expand Down
12 changes: 4 additions & 8 deletions .github/workflows/ci-shark-ai.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,16 +55,12 @@ jobs:
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -r requirements.txt -e sharktank/ shortfin/

# Install latest iree-tubrine.
pip install --no-compile -f https://iree.dev/pip-release-links.html --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"

# Try with the latest IREE nightly releases, not what iree-turbine pins.
# Install nightly IREE packages.
# We could also pin to a known working or stable version.
# This should eventually stabilize. Do the best we can for now.
pip install -f https://iree.dev/pip-release-links.html --upgrade --pre \
pip install -f https://iree.dev/pip-release-links.html --pre --upgrade \
iree-base-compiler \
iree-base-runtime
iree-base-runtime \
iree-turbine

pip freeze

Expand Down
45 changes: 22 additions & 23 deletions .github/workflows/ci-sharktank.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,12 @@ jobs:
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -r requirements.txt -r sharktank/requirements-tests.txt -e sharktank/

# Update to the latest iree packages.
pip install -f https://iree.dev/pip-release-links.html --upgrade --pre \
iree-base-compiler iree-base-runtime --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"
# Install nightly IREE packages.
# We could also pin to a known working or stable version.
pip install -f https://iree.dev/pip-release-links.html --pre \
iree-base-compiler \
iree-base-runtime \
iree-turbine

- name: Run sharktank tests
if: ${{ !cancelled() }}
Expand All @@ -83,7 +85,7 @@ jobs:
run:
shell: bash
env:
PIP_CACHE_DIR: "${{ github.workspace }}/.pip-cache"
VENV_DIR: ${{ github.workspace }}/.venv
HF_HOME: "/data/huggingface"
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
Expand All @@ -94,35 +96,29 @@ jobs:
with:
python-version: ${{matrix.version}}

- name: Cache Pip Packages
uses: actions/cache@6849a6489940f00c2f30c0fb92c6274307ccb58a # v4.1.2
id: cache-pip
with:
path: ${{ env.PIP_CACHE_DIR }}
key: pip-${{ steps.setup_python.outputs.python-version }}-${{ hashFiles('*requirements*.txt','sharktank/requirements*.txt') }}
- name: Create Python venv
run: python -m venv ${VENV_DIR}

- name: Install sharktank deps
run: |
source ${VENV_DIR}/bin/activate
python -m pip install --no-compile --upgrade pip
# Note: We install in three steps in order to satisfy requirements
# from non default locations first. Installing the PyTorch CPU
# wheels saves multiple minutes and a lot of bandwidth on runner setup.
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -r requirements.txt -r sharktank/requirements-tests.txt -e sharktank/

# Install latest iree-tubrine.
pip install --no-compile -f https://iree.dev/pip-release-links.html --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"

# Try with the latest IREE nightly releases, not what iree-turbine pins.
# Install nightly IREE packages.
# We could also pin to a known working or stable version.
# This should eventually stabilize. Do the best we can for now.
pip install -f https://iree.dev/pip-release-links.html --upgrade --pre \
pip install -f https://iree.dev/pip-release-links.html --pre --upgrade \
iree-base-compiler \
iree-base-runtime
iree-base-runtime \
iree-turbine

- name: Run tests
run: |
source ${VENV_DIR}/bin/activate
pytest \
--with-t5-data \
sharktank/tests/models/t5/t5_test.py \
Expand Down Expand Up @@ -158,10 +154,13 @@ jobs:
# wheels saves multiple minutes and a lot of bandwidth on runner setup.
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -r requirements.txt -r sharktank/requirements-tests.txt -e sharktank/
# Update to the latest iree packages.
pip install -f https://iree.dev/pip-release-links.html --upgrade --pre \
iree-base-compiler iree-base-runtime --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"

# Install nightly IREE packages.
# We could also pin to a known working or stable version.
pip install -f https://iree.dev/pip-release-links.html --pre --upgrade \
iree-base-compiler \
iree-base-runtime \
iree-turbine
- name: Run punet tests
run: |
pytest -v sharktank/ -m punet_quick \
Expand Down
19 changes: 8 additions & 11 deletions .github/workflows/ci_eval.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,16 +57,12 @@ jobs:
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -r requirements.txt -r sharktank/requirements-tests.txt -e sharktank/

# Install latest iree-tubrine.
pip install --no-compile -f https://iree.dev/pip-release-links.html --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"

# Try with the latest IREE nightly releases, not what iree-turbine pins.
# Install nightly IREE packages.
# We could also pin to a known working or stable version.
# This should eventually stabilize. Do the best we can for now.
pip install -f https://iree.dev/pip-release-links.html --upgrade --pre \
pip install -f https://iree.dev/pip-release-links.html --pre \
iree-base-compiler \
iree-base-runtime
iree-base-runtime \
iree-turbine

pip freeze

Expand Down Expand Up @@ -119,9 +115,10 @@ jobs:
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -r requirements.txt -r sharktank/requirements-tests.txt -e sharktank/

# Install latest iree-tubrine.
pip install --no-compile -f https://iree.dev/pip-release-links.html --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"
# Install nightly iree-turbine.
# We could also pin to a known working or stable version.
pip install -f https://iree.dev/pip-release-links.html --pre --upgrade \
iree-turbine

- name: Run perplexity test with Torch
run: |
Expand Down
12 changes: 4 additions & 8 deletions .github/workflows/ci_eval_short.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,16 +56,12 @@ jobs:
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -r requirements.txt -r sharktank/requirements-tests.txt -e sharktank/

# Install latest iree-tubrine.
pip install --no-compile -f https://iree.dev/pip-release-links.html --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"

# Try with the latest IREE nightly releases, not what iree-turbine pins.
# Install nightly IREE packages.
# We could also pin to a known working or stable version.
# This should eventually stabilize. Do the best we can for now.
pip install -f https://iree.dev/pip-release-links.html --upgrade --pre \
pip install -f https://iree.dev/pip-release-links.html --pre --upgrade \
iree-base-compiler \
iree-base-runtime
iree-base-runtime \
iree-turbine

pip freeze

Expand Down
19 changes: 6 additions & 13 deletions docs/nightly_releases.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,26 +86,19 @@ deactivate

## Installing newer versions of dependencies

To install the `iree-turbine` package from the latest source:

```bash
pip install --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"
```

To install the `iree-base-compiler` and `iree-base-runtime` packages from
nightly releases:
To install all IREE packages from nightly releases:

```bash
pip install -f https://iree.dev/pip-release-links.html --upgrade --pre \
iree-base-compiler iree-base-runtime
iree-base-compiler \
iree-base-runtime \
iree-turbine
```

To install all three packages together:
To install an editable `iree-turbine` package from the latest source:

```bash
pip install -f https://iree.dev/pip-release-links.html --upgrade --pre \
iree-base-compiler iree-base-runtime --src deps \
pip install --src deps \
-e "git+https://github.com/iree-org/iree-turbine.git#egg=iree-turbine"
```

Expand Down
Loading