Skip to content

SocialGouv/vllm-managed-instance

Repository files navigation

getting started

install

pyenv global 3.11
poetry install --no-root

run

poetry run python main.py 

ovh

debugging

resources

generate docker-compose template

install gomplate: https://docs.gomplate.ca/installing/

export SERVICE_REPLICAS=2
export GPU_BY_REPLICA=1
cat templates/docker-compose.tpl | gomplate > docker-compose.yaml

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors 3

  •  
  •  
  •