Skip to content

Commit

Permalink
docs: Added client docs for connecting to Predibase endpoints
Browse files Browse the repository at this point in the history
  • Loading branch information
tgaddair committed Dec 2, 2023
1 parent b66492e commit 86e5156
Showing 1 changed file with 25 additions and 0 deletions.
25 changes: 25 additions & 0 deletions clients/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,31 @@ print(text)
# ' Rayleigh scattering'
```

### Predibase Inference Endpoints

The LoRAX client can also be used to connect to [Predibase](https://predibase.com/) managed LoRAX endpoints (including Predibase's [serverless endpoints](https://docs.predibase.com/user-guide/inference/serverless_deployments)).

You need only make the following changes to the above examples:

1. Change the `endpoint_url` to match the endpoint of your Predibase LLM of choice.
2. Provide your Predibase API token in the `headers` provided to the client.

Example:

```python
from lorax import Client

endpoint_url = f"https://api.predibase.com/v1/llms/{llm_deployment_name}"
headers = {
"Authorization": f"Bearer {api_token}"
}

client = Client(endpoint_url, headers=headers)

# same as above from here ...
response = client.generate("Why is the sky blue?", adapter_id="some/adapter")
```

### Types

```python
Expand Down

0 comments on commit 86e5156

Please sign in to comment.