Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add component list and optimize image tag step #11

Merged
merged 2 commits into from
May 6, 2024
Merged

add component list and optimize image tag step #11

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8bf0798
add component list and optimize image tag step
jeremyeder May 6, 2024
7a70eb7
Update dictionary for spellcheck
russellb May 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .spellcheck-en-custom.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ codebases
compositional
Conala
Corbett
CPUs
curation
cybette
danmcp
Expand All @@ -60,6 +61,7 @@ datasets
dave
DCO
De
deepspeed
deployable
DeSaix
dev
Expand Down
25 changes: 14 additions & 11 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,15 @@ simplification, as well as a broadening of capabilities, hardware and software
support versions, performance improvements (and other optimizations) prior to
GA.

RHEL AI is an open-source product that includes:

- [Granite](https://huggingface.co/instructlab): an open source, Apache 2 licensed foundation model from IBM.
- [InstructLab](https://github.com/instructlab): a CLI and tuning backend that provides a simple user interface for contributing knowledge and skills to a base model.
- [RHEL Image Mode (bootc)](https://github.com/containers/bootc): RHEL AI is distributed as a “bootable container” image. Provision RHEL AI appliances via kickstart onto bare metal or cloud instances.
- [vLLM](https://github.com/vllm-project/vllm): A high-throughput and memory-efficient inference and serving engine for LLMs, based on PyTorch.
- [deepspeed](https://www.deepspeed.ai/): A deep learning optimization software suite for both training and inference.
- [PyTorch](https://pytorch.org/): PyTorch is an optimized tensor library for deep learning using GPUs and CPUs.

> [!NOTE]
> RHEL AI is targeted at server platforms and workstations with discrete GPUs.
> For laptops, please use upstream [InstructLab](https://github.com/instructlab).
Expand Down Expand Up @@ -157,24 +166,18 @@ Image-mode “bootable” container. We embed the 3 images above into this
container.

```sh
make nvidia FROM=registry.redhat.io/rhel9/rhel-bootc:9.4
make nvidia FROM=registry.redhat.io/rhel9/rhel-bootc:9.4 REGISTRY=<your-registry> REGISTRY_ORG=<your-org-name>
```

The resulting image is tagged `quay.io/rhelai-dev-preview/nvidia-bootc:latest`.
The resulting image is tagged `${REGISTRY}/${REGISTRY_ORG}/nvidia-bootc:latest`.
For more variables and examples, see the
[training/README](https://github.com/rhelai-dev-preview/tree/main/training).

Tag your image with your registry name and path:

```sh
podman tag quay.io/<your-user-name>/nvidia-bootc:latest quay.io/<your-user-name>/nvidia-bootc:latest
```

Push the resulting image to your registry. You will refer to this URL inside a
kickstart file in an upcoming step.
Push the resulting image to your registry. You will refer to this URL inside a kickstart file in an upcoming step.

```sh
podman push quay.io/<your-user-name>/nvidia-bootc:latest
podman push ${REGISTRY}/${REGISTRY_ORG}/nvidia-bootc:latest
e.g. podman push quay.io/<your-user-name>/nvidia-bootc.latest
```

> At this point you have a RHEL AI bootable container image ready to be installed on a physical or virtual host.
Expand Down