Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2 hardware agnostic front and backend #5

Open
wants to merge 36 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
070db5c
'Add AMD support for TorchServe'
smedegaard Nov 1, 2024
ce19723
Update README.md with rocm flags
smedegaard Nov 11, 2024
0fad8e2
add rocm to CONTRIBUTING.md
smedegaard Nov 11, 2024
3247498
WorkerLifeCycle uses SystemInfo to get X_VISIBLE_DEVICES
smedegaard Nov 12, 2024
bae9b2c
AppleUtil adds Accelerator `number_of_cores` times
smedegaard Nov 12, 2024
88f3cb8
fix typo in README.md
smedegaard Nov 13, 2024
8e4d24c
remove mention of java version from README.md
smedegaard Nov 13, 2024
ff4daa8
revert unnecessary changes
samutamm Nov 14, 2024
0bc3e3c
Fix import errors in AppleUtils
jakki-amd Nov 14, 2024
1e635e1
remove rocm support from dockerfile.dev to simplify
samutamm Nov 14, 2024
1647826
fix missing newline
samutamm Nov 14, 2024
0dc5145
revert unnecessary changes
samutamm Nov 14, 2024
f905d0e
'improve formatting for amd_support.md'
Nov 14, 2024
9a515b8
Fix AppleUtils tests
jakki-amd Nov 18, 2024
9d30159
fixes 11. parse-metrics-failed-collecting-amd-gpu-metrics (#24)
smedegaard Nov 20, 2024
8cdf54b
extend testMetricManager
Nov 20, 2024
bd95835
Merge pull request #25 from nod-ai/9-extend-java-testmetricmanager
eppane Nov 21, 2024
e5d382f
Add latest ROCM support
Nov 14, 2024
607d836
Merge pull request #26 from nod-ai/19-add-support-for-latest-torch-rocm
jakki-amd Nov 21, 2024
f2d17d5
PR 24 system_metrics bugfix
Nov 22, 2024
49bc051
Format files
jakki-amd Nov 22, 2024
4bff6d3
Update docs/hardware_support/amd_support.md
smedegaard Nov 26, 2024
b9a1627
typo in docs/hardware_support/amd_support.md
smedegaard Nov 26, 2024
964e5f1
Update docs/hardware_support/amd_support.md
smedegaard Nov 26, 2024
61da32e
Update docs/hardware_support/amd_support.md
smedegaard Nov 26, 2024
0a4d628
remove pyrsmi and nvgpu deps
Nov 26, 2024
aa96f2f
metric collector revert gpu arg name
Nov 26, 2024
a26eefb
fix number of metrics assertion in testMetricManager
Nov 26, 2024
f0b1dfb
'move Intel docs under Hardware Support' (#31)
smedegaard Nov 27, 2024
d330494
Fix docstring
jakki-amd Nov 27, 2024
cbdfe25
Add Dockerfile.rocm
jakki-amd Nov 28, 2024
8330233
Remove sharing lock from bind mounts
jakki-amd Nov 28, 2024
9e5afd0
Update Dockerfile.rocm
jakki-amd Nov 29, 2024
8f35524
Revert Dockerfile changes
jakki-amd Nov 29, 2024
f5ce2ec
Update documentation for Docker support
jakki-amd Nov 29, 2024
f03d0fd
Merge branch 'master' into 2-hardware-agnostic-front-and-backend
jakki-amd Nov 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,9 @@ instances.yaml.backup
# cpp
cpp/_build
cpp/third-party

# projects
.tool-versions
**/*/.classpath
**/*/.settings
**/*/.project
57 changes: 25 additions & 32 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,7 @@ Your contributions will fall into two categories:
- Search for your issue here: https://github.com/pytorch/serve/issues (look for the "good first issue" tag if you're a first time contributor)
- Pick an issue and comment on the task that you want to work on this feature.
- To ensure your changes doesn't break any of the existing features run the sanity suite as follows from serve directory:
- Install dependencies (if not already installed)
For CPU

```bash
python ts_scripts/install_dependencies.py --environment=dev
smedegaard marked this conversation as resolved.
Show resolved Hide resolved
```

For GPU
```bash
python ts_scripts/install_dependencies.py --environment=dev --cuda=cu121
```
> Supported cuda versions as cu121, cu118, cu117, cu116, cu113, cu111, cu102, cu101, cu92
- [Install dependencies](#Install-TorchServe-for-development) (if not already installed)
- Install `pre-commit` to your Git flow:
```bash
pre-commit install
Expand Down Expand Up @@ -60,26 +49,30 @@ pytest -k test/pytest/test_mnist_template.py

If you plan to develop with TorchServe and change some source code, you must install it from source code.

Ensure that you have `python3` installed, and the user has access to the site-packages or `~/.local/bin` is added to the `PATH` environment variable.

Run the following script from the top of the source directory.

NOTE: This script force re-installs `torchserve`, `torch-model-archiver` and `torch-workflow-archiver` if existing installations are found

#### For Debian Based Systems/ MacOS

```
python ./ts_scripts/install_dependencies.py --environment=dev
python ./ts_scripts/install_from_src.py --environment=dev
```

Use `--cuda` flag with `install_dependencies.py` for installing cuda version specific dependencies. Possible values are `cu111`, `cu102`, `cu101`, `cu92`

#### For Windows

Refer to the documentation [here](docs/torchserve_on_win_native.md).

For information about the model archiver, see [detailed documentation](model-archiver/README.md).
1. Clone the repository, including third-party modules, with `git clone --recurse-submodules --remote-submodules [email protected]:pytorch/serve.git`
eppane marked this conversation as resolved.
Show resolved Hide resolved
2. Ensure that you have `python3` installed, and the user has access to the site-packages or `~/.local/bin` is added to the `PATH` environment variable.
3. Run the following script from the top of the source directory. NOTE: This script force re-installs `torchserve`, `torch-model-archiver` and `torch-workflow-archiver` if existing installations are found

#### For Debian Based Systems/MacOS

```
python ./ts_scripts/install_dependencies.py --environment=dev
python ./ts_scripts/install_from_src.py --environment=dev
```
##### Installing Dependencies for Accelerator Support
Use the optional `--rocm` or `--cuda` flag with `install_dependencies.py` for installing accelerator specific dependencies.

Possible values are
- rocm: `rocm61`, `rocm60`
- cuda: `cu111`, `cu102`, `cu101`, `cu92`

For example `python ./ts_scripts/install_dependencies.py --environment=dev --rocm=rocm61`

#### For Windows

Refer to the documentation [here](docs/torchserve_on_win_native.md).

For information about the model archiver, see [detailed documentation](model-archiver/README.md).

### What to Contribute?

Expand Down
10 changes: 8 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,10 @@ curl http://127.0.0.1:8080/predictions/bert -T input.txt

```bash
# Install dependencies
# cuda is optional
python ./ts_scripts/install_dependencies.py

# Include dependencies for accelerator support with the relevant optional flags
python ./ts_scripts/install_dependencies.py --rocm=rocm61
python ./ts_scripts/install_dependencies.py --cuda=cu121

# Latest release
Expand All @@ -36,7 +39,10 @@ pip install torchserve-nightly torch-model-archiver-nightly torch-workflow-archi

```bash
# Install dependencies
# cuda is optional
python ./ts_scripts/install_dependencies.py

# Include depeendencies for accelerator support with the relevant optional flags
smedegaard marked this conversation as resolved.
Show resolved Hide resolved
python ./ts_scripts/install_dependencies.py --rocm=rocm61
python ./ts_scripts/install_dependencies.py --cuda=cu121

# Latest release
Expand Down
Loading
Loading