Skip to content

Commit

Permalink
[Docs] Refactoring Docker Container Docs (#2747)
Browse files Browse the repository at this point in the history 
* upd docs

* Update

* Apply suggestions from code review

Co-authored-by: Zongheng Yang <[email protected]>

* move "Building containers remotely" and add private registry for containerized apps

* add multi-node scenario

* Updates

* add ref link and requirements for image id

---------

Co-authored-by: Zongheng Yang <[email protected]>
  • Loading branch information
@cblmemo @concretevitamin
cblmemo and concretevitamin authored Nov 2, 2023
1 parent da2d6ec commit 53ba415
Showing 1 changed file with 103 additions and 43 deletions.
146 changes: 103 additions & 43 deletions docs/source/examples/docker-containers.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,73 +3,59 @@
Using Docker Containers
=======================

SkyPilot can run a Docker container either as the runtime environment for your task, or as the task itself.
SkyPilot can run a container either as a task, or as the runtime environment of a cluster.

Using Docker Containers as Runtime Environment
----------------------------------------------
* If the container image is invocable / has an entrypoint: run it :ref:`as a task <docker-containers-as-tasks>`.
* Otherwise, the container image is likely to be used as a runtime environment (e.g., ``ubuntu``) and you likely have extra commands to run inside the container: run it :ref:`as a runtime environment <docker-containers-as-runtime-environments>`.

When a container is used as the runtime environment, the SkyPilot task is executed inside the container.
.. _docker-containers-as-tasks:

This means all :code:`setup` and :code:`run` commands in the YAML file will be executed in the container, and any files created by the task will be stored inside the container.
Any GPUs assigned to the task will be automatically mapped to your Docker container and all future tasks on the cluster will also execute in the container.
Running Containers as Tasks
---------------------------

To use a Docker image as your runtime environment, set the :code:`image_id` field in the :code:`resources` section of your task YAML file to :code:`docker:<image_id>`.
For example, to use the :code:`ubuntu:20.04` image from Docker Hub:

.. code-block:: yaml
resources:
image_id: docker:ubuntu:20.04
SkyPilot can run containerized applications directly as regular tasks. The default VM images provided by SkyPilot already have the Docker runtime pre-configured.

setup: |
# Will run inside container
run: |
# Will run inside container
To launch a containerized application, you can directly invoke :code:`docker run` in the :code:`run` section of your task.

For Docker images hosted on private registries, you can provide the registry authentication details using :ref:`task environment variables <env-vars>`:
For example, to run a HuggingFace TGI serving container:

.. code-block:: yaml
# ecr_private_docker.yaml
resources:
image_id: docker:<your-user-id>.dkr.ecr.us-east-1.amazonaws.com/<your-private-image>:<tag>
# the following shorthand is also supported:
# image_id: docker:<your-private-image>:<tag>
envs:
SKYPILOT_DOCKER_USERNAME: AWS
# SKYPILOT_DOCKER_PASSWORD: <password>
SKYPILOT_DOCKER_SERVER: <your-user-id>.dkr.ecr.us-east-1.amazonaws.com
accelerators: A100:1
We suggest setting the :code:`SKYPILOT_DOCKER_PASSWORD` environment variable through the CLI (see :ref:`passing secrets <passing-secrets>`):

.. code-block:: console
$ export SKYPILOT_DOCKER_PASSWORD=$(aws ecr get-login-password --region us-east-1)
$ sky launch ecr_private_docker.yaml --env SKYPILOT_DOCKER_PASSWORD
Running Docker Containers as Tasks
----------------------------------
run: |
docker run --gpus all --shm-size 1g -v ~/data:/data \
ghcr.io/huggingface/text-generation-inference \
--model-id lmsys/vicuna-13b-v1.5
As an alternative, SkyPilot can run docker containers as tasks. Docker runtime is configured and ready for use on the default VM image used by SkyPilot.
# NOTE: Uncommon to have any commands after the above.
# `docker run` is blocking, so any commands after it
# will NOT be run inside the container.
To run a container as a task, you can directly invoke the :code:`docker run` command in the :code:`run` section of your task.
Private Registries
^^^^^^^^^^^^^^^^^^

For example, to run a GPU-accelerated container that prints the output of :code:`nvidia-smi`:
When using this mode, to access Docker images hosted on private registries,
simply add a :code:`setup` section to your task YAML file to authenticate with
the registry:

.. code-block:: yaml
resources:
accelerators: V100:1
accelerators: A100:1
setup: |
# Authenticate with private registry
docker login -u <username> -p <password> <registry>
run: |
docker run --rm --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
docker run <registry>/<image>:<tag>
Building containers remotely
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If you are running the container as a task, the container image can also be built remotely on the cluster in the :code:`setup` phase of the task.
If you are running containerized applications, the container image can also be built remotely on the cluster in the :code:`setup` phase of the task.

The :code:`echo_app` `example <https://github.com/skypilot-org/skypilot/tree/master/examples/docker>`_ provides an example on how to do this:

Expand Down Expand Up @@ -100,3 +86,77 @@ The inputs to the app are copied to SkyPilot using :code:`file_mounts` and mount
The output of the app produced at :code:`/outputs` path in the container is also volume mounted to :code:`/outputs` on the VM, which gets directly written to a S3 bucket through SkyPilot Storage mounting.

Our GitHub repository has more examples, including running `Detectron2 in a Docker container <https://github.com/skypilot-org/skypilot/blob/master/examples/detectron2_docker.yaml>`_ via SkyPilot.

.. _docker-containers-as-runtime-environments:

Using Containers as Runtime Environments
----------------------------------------

When a container is used as the runtime environment, everything happens inside the container:

- The SkyPilot runtime is automatically installed and launched inside the container;
- :code:`setup` and :code:`run` commands are executed in the container;
- Any files created by the task will be stored inside the container.

To use a Docker image as your runtime environment, set the :code:`image_id` field in the :code:`resources` section of your task YAML file to :code:`docker:<image_id>`.
For example, to use the :code:`ubuntu:20.04` image from Docker Hub:

.. code-block:: yaml
resources:
image_id: docker:ubuntu:20.04
setup: |
# Commands to run inside the container
run: |
# Commands to run inside the container
Any GPUs assigned to the task will be automatically mapped to your Docker container, and all subsequent tasks within the cluster will also run inside the container. In a multi-node scenario, the container will be launched on all nodes, and the corresponding node's container will be assigned for task execution.

.. tip::

**When to use this?**

If you have a preconfigured development environment set up within a Docker
image, it can be convenient to use the runtime environment mode. This is
especially useful for launching development environments that are
challenging to configure on a new virtual machine, such as dependencies on
specific versions of CUDA or cuDNN.

.. note::

Since we ``pip install skypilot`` inside the user-specified container image
as part of a launch, users should ensure dependency conflicts do not occur.

Currently, the following requirements must be met:

1. The container image should be based on Debian;

2. The container image must grant sudo permissions without requiring password authentication for the user. Having a root user is also acceptable.

Private Registries
^^^^^^^^^^^^^^^^^^

When using this mode, to access Docker images hosted on private registries,
you can provide the registry authentication details using :ref:`task environment variables <env-vars>`:

.. code-block:: yaml
# ecr_private_docker.yaml
resources:
image_id: docker:<your-user-id>.dkr.ecr.us-east-1.amazonaws.com/<your-private-image>:<tag>
# the following shorthand is also supported:
# image_id: docker:<your-private-image>:<tag>
envs:
SKYPILOT_DOCKER_USERNAME: AWS
# SKYPILOT_DOCKER_PASSWORD: <password>
SKYPILOT_DOCKER_SERVER: <your-user-id>.dkr.ecr.us-east-1.amazonaws.com
We suggest setting the :code:`SKYPILOT_DOCKER_PASSWORD` environment variable through the CLI (see :ref:`passing secrets <passing-secrets>`):

.. code-block:: console
$ export SKYPILOT_DOCKER_PASSWORD=$(aws ecr get-login-password --region us-east-1)
$ sky launch ecr_private_docker.yaml --env SKYPILOT_DOCKER_PASSWORD

0 comments on commit 53ba415

Please sign in to comment.