Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update client docs with new endpoint source #126

Merged
merged 5 commits into from
Dec 21, 2023
Merged

Update client docs with new endpoint source #126

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6bed187
Update client docs with new endpoint source
Dec 13, 2023
abdd7c9
update with instructions on tenant short code
Dec 14, 2023
086553a
update docs
magdyksaleh Dec 20, 2023
be6da24
update docs and client
magdyksaleh Dec 20, 2023
eb16690
update docs
Dec 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion clients/python/lorax/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,9 @@ def generate(
payload = resp.json()
if resp.status_code != 200:
raise parse_error(resp.status_code, payload)
return Response(**payload[0])
if isinstance(payload, list):
return Response(**payload[0])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This smells like bad design IMO. The REST API should be consistent and predictable. We should not be writing client-side hacks to account for inconsistency in the REST API.

return Response(**payload)

def generate_stream(
self,
Expand Down
4 changes: 3 additions & 1 deletion docs/reference/python_client.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,9 @@ Example:
```python
from lorax import Client

endpoint_url = f"https://api.app.predibase.com/v1/llms/{llm_deployment_name}"
# You can get your Predibase API token by going to Settings > My Profile > Generate API Token
# You can get your Predibase Tenant short code by going to Settings > My Profile > Overview > Tenant ID
endpoint_url = f"https://serving.app.predibase.com/{predibase_tenant_short_code}/deployments/v2/llms/{llm_deployment_name}/"
headers = {
"Authorization": f"Bearer {api_token}"
}
Expand Down
Loading