Skip to content

Commit

Permalink
update README
Browse files Browse the repository at this point in the history
  • Loading branch information
samos123 committed Oct 28, 2023
1 parent 2531d09 commit a2cdb0f
Showing 1 changed file with 31 additions and 8 deletions.
39 changes: 31 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,25 @@
# Sentence Transformers API

OpenAI compatible embedding API that uses Sentence Transformer for embedding models
OpenAI compatible embedding API that uses Sentence Transformer for embeddings

Support the project by adding a star! ❤️
Container Image: `ghcr.io/substratusai/sentence-transformers-api`

Support the project by adding a star! ❤️
Join us on Discord:
<a href="https://discord.gg/JeXhcmjZVm">
<img alt="discord-invite" src="https://dcbadge.vercel.app/api/server/JeXhcmjZVm?style=flat">
</a>

## Install (Docker)
## Install
There are 2 options to install: Docker or local Python install.

### Install (Docker)
Run the API locally using Docker:
```
```bash
docker run -p 8080:8080 -d ghcr.io/substratusai/sentence-transformers-api
```

## Install (Local python)
### Install (Local python)
Install and run the API server locally using Python. Only supports python 3.9, 3.10 and 3.11.

Clone the repo:
Expand All @@ -30,12 +34,13 @@ pip3 install -r requirements.txt
```

Run the webserver:
```
```bash
uvicorn main:app --port 8080 --reload
```

## Usage
Now you can visit the API docs on [http://localhost:8080/docs](http://localhost:8080/docs)
After you've installed sentence-transformers-api,
you can visit the API docs on [http://localhost:8080/docs](http://localhost:8080/docs)

You can also use CURL to get embeddings:
```bash
Expand All @@ -57,12 +62,30 @@ embedding = openai.Embedding.create(input="Some text", model=model)["data"][0]["
print(embedding)
```

## Supported Models
Any model that's supported by Sentence Transformers should also work as-is
with Sentence Transformers API.
Here is a list of [pre-trained models](https://www.sbert.net/docs/pretrained_models.html) available with Sentence Transformers.

By default the `all-MiniLM-L6-v2` model is used and preloaded on startup. You
can preload any supported model by setting the `MODEL` environment variable.

For example, if you want to preload the `multi-qa-MiniLM-L6-cos-v1`, you
could tweak the `docker run` command like this:
```bash
docker run -e MODEL=multi-qa-MiniLM-L6-cos-v1 -p 8080:8080 -d \
ghcr.io/substratusai/sentence-transformers-api
```

In addition to preloading models, you can also specify other models at runtime. A single
API endpoint can serve multiple models in parallel.

## Integrations
It's easy to utilize the embedding server with various other tools because
the API is compatible with the OpenAI Embedding API.

### Weaviate
TODO Write weaviate guide here
TODO Write Weaviate guide here


## Creators
Expand Down

0 comments on commit a2cdb0f

Please sign in to comment.