diff --git a/README.md b/README.md index 4e0d7e6..3202969 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,19 @@ -![happy_vllm logo](https://raw.githubusercontent.com/OSS-Pole-Emploi/happy_vllm/main/docs/source/assets/logo/logo_happy_vllm.svg) +![happy_vllm logo](https://raw.githubusercontent.com/France-Travail/happy_vllm/main/docs/source/assets/logo/logo_happy_vllm.svg) [![pypi badge](https://img.shields.io/pypi/v/happy_vllm.svg)](https://pypi.python.org/pypi/happy_vllm) [![Generic badge](https://img.shields.io/badge/python-3.10|3.11-blue.svg)](https://shields.io/) [![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0) -![Build & Tests](https://github.com/OSS-Pole-Emploi/happy_vllm/actions/workflows/build_and_tests.yaml/badge.svg) -![Wheel setup](https://github.com/OSS-Pole-Emploi/happy_vllm/actions/workflows/wheel.yaml/badge.svg) -![docs](https://github.com/OSS-Pole-Emploi/happy_vllm/actions/workflows/docs.yaml/badge.svg) +![Build & Tests](https://github.com/France-Travail/happy_vllm/actions/workflows/build_and_tests.yaml/badge.svg) +![Wheel setup](https://github.com/France-Travail/happy_vllm/actions/workflows/wheel.yaml/badge.svg) +![docs](https://github.com/France-Travail/happy_vllm/actions/workflows/docs.yaml/badge.svg) -**📚 Documentation :** [https://oss-pole-emploi.github.io/happy_vllm/](https://oss-pole-emploi.github.io/happy_vllm/) +**📚 Documentation :** [https://france-travail.github.io/happy_vllm/](https://france-travail.github.io/happy_vllm/) --- -happy_vLLM is a REST API for [vLLM](https://github.com/vllm-project/vllm) which was developed with production in mind. It adds some [functionalities](https://oss-pole-emploi.github.io/happy_vllm/pros/) to vLLM. +happy_vLLM is a REST API for [vLLM](https://github.com/vllm-project/vllm) which was developed with production in mind. It adds some [functionalities](https://france-travail.github.io/happy_vllm/pros/) to vLLM. ## Installation @@ -26,14 +26,14 @@ pip install happy_vllm Or build it from source: ```bash -git clone https://github.com/OSS-Pole-Emploi/happy_vllm.git +git clone https://github.com/France-Travail/happy_vllm.git cd happy_vllm pip install -e . ``` ## Quickstart -Just use the entrypoint `happy-vllm` (see [arguments](https://oss-pole-emploi.github.io/happy_vllm/arguments/) for a list of all possible arguments) +Just use the entrypoint `happy-vllm` (see [arguments](https://france-travail.github.io/happy_vllm/arguments/) for a list of all possible arguments) ```bash happy_vllm --model path_to_model --host 127.0.0.1 --port 5000 --model-name my_model @@ -51,16 +51,16 @@ To get various information on the application or curl 127.0.0.1:5000/v1/completions -d '{"prompt": "Hey,", "model": "my_model"}' ``` -if you want to generate your first LLM response using happy_vLLM. See [endpoints](https://oss-pole-emploi.github.io/happy_vllm/endpoints/endpoints) for more details on all the endpoints provided by happy_vLLM. +if you want to generate your first LLM response using happy_vLLM. See [endpoints](https://france-travail.github.io/happy_vllm/endpoints/endpoints) for more details on all the endpoints provided by happy_vLLM. ## Deploy with Docker image -A docker image is available from the [Github Container Registry](https://github.com/OSS-Pole-Emploi/happy_vllm/pkgs/container/happy_vllm) : +A docker image is available from the [Github Container Registry](https://github.com/France-Travail/happy_vllm/pkgs/container/happy_vllm) : ```bash -docker pull ghcr.io/oss-pole-emploi/happy_vllm:latest +docker pull ghcr.io/france-travail/happy_vllm:latest ``` -See [deploying_with_docker](https://oss-pole-emploi.github.io/happy_vllm/deploying_with_docker) for more details on how to serve happy_vLLM with docker. +See [deploying_with_docker](https://france-travail.github.io/happy_vllm/deploying_with_docker) for more details on how to serve happy_vLLM with docker. ## Swagger diff --git a/docs/deploying_with_docker.md b/docs/deploying_with_docker.md index e3c8414..ef435e6 100644 --- a/docs/deploying_with_docker.md +++ b/docs/deploying_with_docker.md @@ -1,22 +1,22 @@ # Deploying with docker -The docker image is available on the [Github Container Registry](https://github.com/OSS-Pole-Emploi/happy_vllm/pkgs/container/happy_vllm) +The docker image is available on the [Github Container Registry](https://github.com/France-Travail/happy_vllm/pkgs/container/happy_vllm) ## Pull the image from Github Container Registry ```bash -docker pull ghcr.io/oss-pole-emploi/happy_vllm:latest +docker pull ghcr.io/france-travail/happy_vllm:latest ``` ## Launch a container ```bash -docker run -it ghcr.io/oss-pole-emploi/happy_vllm:latest --model mistralai/Mistral-7B-v0.1 +docker run -it ghcr.io/france-travail/happy_vllm:latest --model mistralai/Mistral-7B-v0.1 ``` See [arguments](arguments.md) for more details the list of all arguments useful for the application and model for happy_vLLM. ## Build docker image from source via the provided dockerfile ```bash -docker build -t oss-pole-emploi/happy_vllm:latest . +docker build -t france-travail/happy_vllm:latest . ``` \ No newline at end of file diff --git a/docs/endpoints/technical.md b/docs/endpoints/technical.md index 0f9dcc8..1cf815a 100644 --- a/docs/endpoints/technical.md +++ b/docs/endpoints/technical.md @@ -49,4 +49,4 @@ The Open AI compatible endpoint used, for example, to get the name of the model. ## /v1/launch_arguments (GET) -If when launched the arguments `--with-launch-arguments` is given, this route gives all the parameters given when launching the application. In particular, the arguments given to the vLLM engine. It is useful for the benchmarks. \ No newline at end of file +If when launched the arguments `--with-launch-arguments` is given, this route gives all the parameters given when launching the application. In particular, the arguments given to the vLLM engine. It is useful for the benchmarks done by [benchmark_llm_serving](https://github.com/France-Travail/benchmark_llm_serving) \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index b3fc5d6..77837c6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -13,7 +13,7 @@ pip install happy_vllm Or build it from source: ```bash -git clone https://github.com/OSS-Pole-Emploi/happy_vllm.git +git clone https://github.com/France-Travail/happy_vllm.git cd happy_vllm pip install -e . ``` @@ -42,10 +42,10 @@ if you want to generate your first LLM response using happy_vLLM. See [endpoints ## Deploy with Docker image -A docker image is available from the [Github Container Registry](https://github.com/OSS-Pole-Emploi/happy_vllm/pkgs/container/happy_vllm) : +A docker image is available from the [Github Container Registry](https://github.com/France-Travail/happy_vllm/pkgs/container/happy_vllm) : ```bash -docker pull ghcr.io/oss-pole-emploi/happy_vllm:latest +docker pull ghcr.io/france-travail/happy_vllm:latest ``` See [deploying_with_docker](deploying_with_docker.md) for more details on how to serve happy_vLLM with docker. diff --git a/docs/pros.md b/docs/pros.md index 7da652b..7987bce 100644 --- a/docs/pros.md +++ b/docs/pros.md @@ -15,4 +15,8 @@ If you would like to see an endpoint added, don't hesitate to open an issue or a ## Swagger -A well documented swagger (the UI being reachable at the `/docs` endpoint) in order for users not so used to using API to be able to quickly get the hang of it and be as autonomous as possible in querying the LLM. \ No newline at end of file +A well documented swagger (the UI being reachable at the `/docs` endpoint) in order for users not so used to using API to be able to quickly get the hang of it and be as autonomous as possible in querying the LLM. + +## Benchmarks + +We developped a library [benchmark_llm_serving](https://github.com/France-Travail/benchmark_llm_serving) which provides a more complete benchmark of the vLLM serving API than the vanilla one. \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index adb9bc1..13c4c6a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,6 +1,6 @@ site_name: happy-vllm documentation -repo_url: https://github.com/OSS-Pole-Emploi/happy_vllm -repo_name: OSS-Pole-Emploi/happy_vllm +repo_url: https://github.com/France-Travail/happy_vllm +repo_name: France-Travail/happy_vllm # see https://squidfunk.github.io/mkdocs-material/setup/ theme: diff --git a/pyproject.toml b/pyproject.toml index dbc68f1..5656a5a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -31,8 +31,8 @@ keywords = ["production", "transformers", "api-rest", "serving", "mlops", "llm", dynamic = ["version"] [project.urls] -Repository = "https://github.com/OSS-Pole-Emploi/happy_vllm" -Documentation = "https://oss-pole-emploi.github.io/happy_vllm" +Repository = "https://github.com/France-Travail/happy_vllm" +Documentation = "https://france-travail.github.io/happy_vllm" [tool.setuptools.dynamic] version = {file = "version.txt"} diff --git a/src/happy_vllm/routers/schemas/examples/request.json b/src/happy_vllm/routers/schemas/examples/request.json index 925e285..8ad4f98 100644 --- a/src/happy_vllm/routers/schemas/examples/request.json +++ b/src/happy_vllm/routers/schemas/examples/request.json @@ -218,7 +218,7 @@ "tokenizer": { "nominal_case": { "summary": "Nominal case", - "description": "This is a nominal case. For more details on what is the vanilla version of the tokenizer or not, please read the documentation : https://oss-pole-emploi.github.io/happy_vllm/", + "description": "This is a nominal case. For more details on what is the vanilla version of the tokenizer or not, please read the documentation : https://france-travail.github.io/happy_vllm/", "value": { "text": "This is a text example", "with_tokens_str": true, @@ -229,7 +229,7 @@ "decode": { "nominal_case": { "summary": "Nominal case", - "description": "Decode a list of token ids. It is the inverse of the endpoint tokenizer. For more details on what is the vanilla version of the tokenizer or not, please read the documentation : https://oss-pole-emploi.github.io/happy_vllm/", + "description": "Decode a list of token ids. It is the inverse of the endpoint tokenizer. For more details on what is the vanilla version of the tokenizer or not, please read the documentation : https://france-travail.github.io/happy_vllm/", "value": { "token_ids": [ 1,