Skip to content

Commit

Permalink
Apply reviews pt1
Browse files Browse the repository at this point in the history
Signed-off-by: davidmirror-ops <[email protected]>
  • Loading branch information
davidmirror-ops committed Jul 24, 2024
1 parent b81b19f commit a181fe6
Showing 1 changed file with 11 additions and 19 deletions.
30 changes: 11 additions & 19 deletions docs/user_guide/productionizing/configuring_access_to_gpus.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

Along with compute resources like CPU and memory, you may want to configure and access GPU resources.

Flyte provides different ways to request accelerator resources directly from the task decorator. This page covers the requirements and procedures to leverage them.
Flyte provides different ways to request accelerator resources directly from the task decorator.

>The examples in this section use [ImageSpec](https://docs.flyte.org/en/latest/user_guide/customizing_dependencies/imagespec.html#imagespec), a Flyte feature that builds a custom container image without a Dockerfile. Install it using `pip install flytekitplugins-envd`.
Expand All @@ -23,7 +23,7 @@ image = ImageSpec(
name="pytorch",
python_version="3.10",
packages=["torch"],
builder="envd",
builder="default",
registry="<YOUR_CONTAINER_REGISTRY>",
)

Expand Down Expand Up @@ -98,7 +98,7 @@ configuration:

## Requesting a specific GPU device

In this example, the goal is to run the task on a specific type of accelerator: NVIDIA Tesla V100 :
The goal is to run the task on a specific type of accelerator: NVIDIA Tesla V100 in the following example:


```python
Expand All @@ -110,7 +110,7 @@ image = ImageSpec(
name="pytorch",
python_version="3.10",
packages=["torch"],
builder="envd",
builder="default",
registry="<YOUR_CONTAINER_REGISTRY>",
)
Expand Down Expand Up @@ -190,7 +190,7 @@ image = ImageSpec(
name="pytorch",
python_version="3.10",
packages=["torch"],
builder="envd",
builder="default",
registry="<YOUR_CONTAINER_REGISTRY>",
)
Expand Down Expand Up @@ -268,7 +268,7 @@ The ``2g.10gb`` value comes from the [NVIDIA A100 supported instance profiles](h

## Additional use cases

### Request an A100 device with no preference for partition configuration
### Request an A100 device with no preference on partition configuration

Example:

Expand All @@ -281,7 +281,7 @@ image = ImageSpec(
name="pytorch",
python_version="3.10",
packages=["torch"],
builder="envd",
builder="default",
registry="<YOUR_CONTAINER_REGISTRY>",
)
Expand All @@ -294,21 +294,13 @@ def gpu_available() -> bool:

#### How it works?

By default, the task is scheduled on a `2g.10gb` MIG partition.
In this case, the task is scheduled with a `2g.10gb` MIG instance profile by default.

`flytepropeller` only injects the node selector that matches nodes with an `A100` device:

```yaml
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: nvidia.com/gpu.accelerator
operator: In
values:
- nvidia-tesla-a100
```


Expand All @@ -324,7 +316,7 @@ image = ImageSpec(
name="pytorch",
python_version="3.10",
packages=["torch"],
builder="envd",
builder="default",
registry="<YOUR_CONTAINER_REGISTRY>",
)
Expand Down

0 comments on commit a181fe6

Please sign in to comment.