Skip to content

Commit

Permalink
Changes resulting from github group migration (#57)
Browse files Browse the repository at this point in the history
  • Loading branch information
gsolard authored and mfournioux committed Jun 27, 2024
1 parent 93acb04 commit 0749c63
Show file tree
Hide file tree
Showing 8 changed files with 31 additions and 27 deletions.
24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
![happy_vllm logo](https://raw.githubusercontent.com/OSS-Pole-Emploi/happy_vllm/main/docs/source/assets/logo/logo_happy_vllm.svg)
![happy_vllm logo](https://raw.githubusercontent.com/France-Travail/happy_vllm/main/docs/source/assets/logo/logo_happy_vllm.svg)

[![pypi badge](https://img.shields.io/pypi/v/happy_vllm.svg)](https://pypi.python.org/pypi/happy_vllm)
[![Generic badge](https://img.shields.io/badge/python-3.10|3.11-blue.svg)](https://shields.io/)
[![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)

![Build & Tests](https://github.com/OSS-Pole-Emploi/happy_vllm/actions/workflows/build_and_tests.yaml/badge.svg)
![Wheel setup](https://github.com/OSS-Pole-Emploi/happy_vllm/actions/workflows/wheel.yaml/badge.svg)
![docs](https://github.com/OSS-Pole-Emploi/happy_vllm/actions/workflows/docs.yaml/badge.svg)
![Build & Tests](https://github.com/France-Travail/happy_vllm/actions/workflows/build_and_tests.yaml/badge.svg)
![Wheel setup](https://github.com/France-Travail/happy_vllm/actions/workflows/wheel.yaml/badge.svg)
![docs](https://github.com/France-Travail/happy_vllm/actions/workflows/docs.yaml/badge.svg)


**📚 Documentation :** [https://oss-pole-emploi.github.io/happy_vllm/](https://oss-pole-emploi.github.io/happy_vllm/) <!-- omit in toc -->
**📚 Documentation :** [https://france-travail.github.io/happy_vllm/](https://france-travail.github.io/happy_vllm/) <!-- omit in toc -->

---

happy_vLLM is a REST API for [vLLM](https://github.com/vllm-project/vllm) which was developed with production in mind. It adds some [functionalities](https://oss-pole-emploi.github.io/happy_vllm/pros/) to vLLM.
happy_vLLM is a REST API for [vLLM](https://github.com/vllm-project/vllm) which was developed with production in mind. It adds some [functionalities](https://france-travail.github.io/happy_vllm/pros/) to vLLM.

## Installation

Expand All @@ -26,14 +26,14 @@ pip install happy_vllm
Or build it from source:

```bash
git clone https://github.com/OSS-Pole-Emploi/happy_vllm.git
git clone https://github.com/France-Travail/happy_vllm.git
cd happy_vllm
pip install -e .
```

## Quickstart

Just use the entrypoint `happy-vllm` (see [arguments](https://oss-pole-emploi.github.io/happy_vllm/arguments/) for a list of all possible arguments)
Just use the entrypoint `happy-vllm` (see [arguments](https://france-travail.github.io/happy_vllm/arguments/) for a list of all possible arguments)

```bash
happy_vllm --model path_to_model --host 127.0.0.1 --port 5000 --model-name my_model
Expand All @@ -51,16 +51,16 @@ To get various information on the application or
curl 127.0.0.1:5000/v1/completions -d '{"prompt": "Hey,", "model": "my_model"}'
```

if you want to generate your first LLM response using happy_vLLM. See [endpoints](https://oss-pole-emploi.github.io/happy_vllm/endpoints/endpoints) for more details on all the endpoints provided by happy_vLLM.
if you want to generate your first LLM response using happy_vLLM. See [endpoints](https://france-travail.github.io/happy_vllm/endpoints/endpoints) for more details on all the endpoints provided by happy_vLLM.

## Deploy with Docker image

A docker image is available from the [Github Container Registry](https://github.com/OSS-Pole-Emploi/happy_vllm/pkgs/container/happy_vllm) :
A docker image is available from the [Github Container Registry](https://github.com/France-Travail/happy_vllm/pkgs/container/happy_vllm) :

```bash
docker pull ghcr.io/oss-pole-emploi/happy_vllm:latest
docker pull ghcr.io/france-travail/happy_vllm:latest
```
See [deploying_with_docker](https://oss-pole-emploi.github.io/happy_vllm/deploying_with_docker) for more details on how to serve happy_vLLM with docker.
See [deploying_with_docker](https://france-travail.github.io/happy_vllm/deploying_with_docker) for more details on how to serve happy_vLLM with docker.

## Swagger

Expand Down
8 changes: 4 additions & 4 deletions docs/deploying_with_docker.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
# Deploying with docker

The docker image is available on the [Github Container Registry](https://github.com/OSS-Pole-Emploi/happy_vllm/pkgs/container/happy_vllm)
The docker image is available on the [Github Container Registry](https://github.com/France-Travail/happy_vllm/pkgs/container/happy_vllm)

## Pull the image from Github Container Registry

```bash
docker pull ghcr.io/oss-pole-emploi/happy_vllm:latest
docker pull ghcr.io/france-travail/happy_vllm:latest
```

## Launch a container

```bash
docker run -it ghcr.io/oss-pole-emploi/happy_vllm:latest --model mistralai/Mistral-7B-v0.1
docker run -it ghcr.io/france-travail/happy_vllm:latest --model mistralai/Mistral-7B-v0.1
```
See [arguments](arguments.md) for more details the list of all arguments useful for the application and model for happy_vLLM.

## Build docker image from source via the provided dockerfile

```bash
docker build -t oss-pole-emploi/happy_vllm:latest .
docker build -t france-travail/happy_vllm:latest .
```
2 changes: 1 addition & 1 deletion docs/endpoints/technical.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,4 @@ The Open AI compatible endpoint used, for example, to get the name of the model.

## /v1/launch_arguments (GET)

If when launched the arguments `--with-launch-arguments` is given, this route gives all the parameters given when launching the application. In particular, the arguments given to the vLLM engine. It is useful for the benchmarks.
If when launched the arguments `--with-launch-arguments` is given, this route gives all the parameters given when launching the application. In particular, the arguments given to the vLLM engine. It is useful for the benchmarks done by [benchmark_llm_serving](https://github.com/France-Travail/benchmark_llm_serving)
6 changes: 3 additions & 3 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ pip install happy_vllm
Or build it from source:

```bash
git clone https://github.com/OSS-Pole-Emploi/happy_vllm.git
git clone https://github.com/France-Travail/happy_vllm.git
cd happy_vllm
pip install -e .
```
Expand Down Expand Up @@ -42,10 +42,10 @@ if you want to generate your first LLM response using happy_vLLM. See [endpoints

## Deploy with Docker image

A docker image is available from the [Github Container Registry](https://github.com/OSS-Pole-Emploi/happy_vllm/pkgs/container/happy_vllm) :
A docker image is available from the [Github Container Registry](https://github.com/France-Travail/happy_vllm/pkgs/container/happy_vllm) :

```bash
docker pull ghcr.io/oss-pole-emploi/happy_vllm:latest
docker pull ghcr.io/france-travail/happy_vllm:latest
```
See [deploying_with_docker](deploying_with_docker.md) for more details on how to serve happy_vLLM with docker.

Expand Down
6 changes: 5 additions & 1 deletion docs/pros.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,8 @@ If you would like to see an endpoint added, don't hesitate to open an issue or a

## Swagger

A well documented swagger (the UI being reachable at the `/docs` endpoint) in order for users not so used to using API to be able to quickly get the hang of it and be as autonomous as possible in querying the LLM.
A well documented swagger (the UI being reachable at the `/docs` endpoint) in order for users not so used to using API to be able to quickly get the hang of it and be as autonomous as possible in querying the LLM.

## Benchmarks

We developped a library [benchmark_llm_serving](https://github.com/France-Travail/benchmark_llm_serving) which provides a more complete benchmark of the vLLM serving API than the vanilla one.
4 changes: 2 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
site_name: happy-vllm documentation
repo_url: https://github.com/OSS-Pole-Emploi/happy_vllm
repo_name: OSS-Pole-Emploi/happy_vllm
repo_url: https://github.com/France-Travail/happy_vllm
repo_name: France-Travail/happy_vllm

# see https://squidfunk.github.io/mkdocs-material/setup/
theme:
Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ keywords = ["production", "transformers", "api-rest", "serving", "mlops", "llm",
dynamic = ["version"]

[project.urls]
Repository = "https://github.com/OSS-Pole-Emploi/happy_vllm"
Documentation = "https://oss-pole-emploi.github.io/happy_vllm"
Repository = "https://github.com/France-Travail/happy_vllm"
Documentation = "https://france-travail.github.io/happy_vllm"

[tool.setuptools.dynamic]
version = {file = "version.txt"}
Expand Down
4 changes: 2 additions & 2 deletions src/happy_vllm/routers/schemas/examples/request.json
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@
"tokenizer": {
"nominal_case": {
"summary": "Nominal case",
"description": "This is a nominal case. For more details on what is the vanilla version of the tokenizer or not, please read the documentation : https://oss-pole-emploi.github.io/happy_vllm/",
"description": "This is a nominal case. For more details on what is the vanilla version of the tokenizer or not, please read the documentation : https://france-travail.github.io/happy_vllm/",
"value": {
"text": "This is a text example",
"with_tokens_str": true,
Expand All @@ -229,7 +229,7 @@
"decode": {
"nominal_case": {
"summary": "Nominal case",
"description": "Decode a list of token ids. It is the inverse of the endpoint tokenizer. For more details on what is the vanilla version of the tokenizer or not, please read the documentation : https://oss-pole-emploi.github.io/happy_vllm/",
"description": "Decode a list of token ids. It is the inverse of the endpoint tokenizer. For more details on what is the vanilla version of the tokenizer or not, please read the documentation : https://france-travail.github.io/happy_vllm/",
"value": {
"token_ids": [
1,
Expand Down

0 comments on commit 0749c63

Please sign in to comment.