After installing Dynamo with the following command, Dynamo can be used primarily through its CLI.
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -yq python3-dev python3-pip python3-venv libucx0
python3 -m venv venv
source venv/bin/activate
pip install ai-dynamo[all]
Dynamo CLI has the following 4 sub-commands.
- 🏃 dynamo run: quickly spin up a server to experiment with a specified model, input and output target.
- 🫴 dynamo serve: compose a graph of workers locally and serve.
- 🔨 (Experimental) dynamo build: containerize either the entire graph or parts of graph to multiple containers
- 🚀 (Experimental) dynamo deploy: deploy to K8 with helm charts or custom operators
- ☁️ (Experimental) dynamo cloud: interact with your dynamo cloud server
For more detailed examples on serving LLMs with disaggregated serving, KV aware routing, etc, please refer to LLM deployment examples