Skip to content

Commit

Permalink
Updated main README
Browse files Browse the repository at this point in the history
  • Loading branch information
tgaddair committed Nov 16, 2023
1 parent 0cd8994 commit de9eee2
Showing 1 changed file with 6 additions and 7 deletions.
13 changes: 6 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,40 +73,39 @@ You can then query the model using either the `/generate` or `/generate_stream`
```shell
curl 127.0.0.1:8080/generate \
-X POST \
-d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":20}}' \
-d '{"inputs":"What is Deep Learning?","parameters":{"adapter_id":"some/adapter"}}' \
-H 'Content-Type: application/json'
```

```shell
curl 127.0.0.1:8080/generate_stream \
-X POST \
-d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":20}}' \
-d '{"inputs":"What is Deep Learning?","parameters":{"adapter_id":"some/adapter"}}' \
-H 'Content-Type: application/json'
```

or from Python:

```shell
pip install lorax
pip install lorax-client
```

```python
from lorax import Client

client = Client("http://127.0.0.1:8080")
print(client.generate("What is Deep Learning?", max_new_tokens=20).generated_text)
print(client.generate("What is Deep Learning?", adapter_id="some/adapter").generated_text)

text = ""
for response in client.generate_stream("What is Deep Learning?", max_new_tokens=20):
for response in client.generate_stream("What is Deep Learning?", adapter_id="some/adapter"):
if not response.token.special:
text += response.token.text
print(text)
```

### 📓 API documentation

You can consult the OpenAPI documentation of the `lorax-inference` REST API using the `/docs` route.
The Swagger UI is also available at: [https://huggingface.github.io/lorax-inference](https://huggingface.github.io/lorax-inference).
You can consult the OpenAPI documentation of the `lorax` REST API using the `/docs` route.

### 🛠️ Local install

Expand Down

0 comments on commit de9eee2

Please sign in to comment.