Skip to content

Commit

Permalink
fix
Browse files Browse the repository at this point in the history
  • Loading branch information
KuilongCui committed Aug 30, 2024
1 parent 7ff05fa commit bc92ef3
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 4 deletions.
4 changes: 2 additions & 2 deletions docs/Quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ cd llumnix
make install
```

If you want to use NCCL as the migration backend, run `make cupy-cuda` to install (cupy-cuda) [https://pypi.org/search/?q=cupy-cuda] manually, as it is related to the CUDA version.
The default migration backend is RPC. If you want to use NCCL as the migration backend, run `make cupy-cuda` to install (cupy-cuda) [https://pypi.org/search/?q=cupy-cuda] manually, as it is related to the CUDA version.

If you want to use Gloo as migration backend, in addition to installing cupy-cuda, please refer to [this link](https://github.com/ZeldaHuang/pygloo/blob/main/.github/workflows/ubuntu_basic.yml#L24C1-L26C1) to install [Bazel](https://github.com/bazelbuild/bazel) >= 5.1.0. Then, run `make pygloo` to install [pygloo](https://github.com/ZeldaHuang/pygloo).
If you want to use Gloo as migration backend, **in addition to installing cupy-cuda**, please refer to [this link](https://github.com/ZeldaHuang/pygloo/blob/main/.github/workflows/ubuntu_basic.yml#L24C1-L26C1) to install [Bazel](https://github.com/bazelbuild/bazel) >= 5.1.0. Then, run `make pygloo` to install [pygloo](https://github.com/ZeldaHuang/pygloo).

Note: Using conda is not recommended, as it cannot properly handle pygloo's dependency on gcc libstdc++.so.6: version GLIBCXX_3.4.30.

Expand Down
4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ def readme():

setup(
name='llumnix',
version='0.0.1',
python_requires='>=3.8.1, <=3.10',
version='0.0.2',
python_requires='>=3.8.1, <3.11',
description='Efficient and easy multi-instance LLM serving',
long_description=readme(),
long_description_content_type="text/markdown",
Expand Down

0 comments on commit bc92ef3

Please sign in to comment.