Update client docs with new endpoint source #126
Conversation
docs/reference/python_client.md
Outdated
| @@ -67,7 +67,7 @@ Example: | |||
| ```python | |||
| from lorax import Client | |||
|
|
|||
| endpoint_url = f"https://api.app.predibase.com/v1/llms/{llm_deployment_name}" | |||
| endpoint_url = f"https://serving.app.predibase.com/{predibase_tenant_id}/llm/llm-deployment-{llm_deployment_name}/generate" | |||
There was a problem hiding this comment.
Can we remove the llm-deployment- when resolving the URL in the backend - we would need to update the VS for this too.
There was a problem hiding this comment.
+1, less verbose the better.
There was a problem hiding this comment.
Can you also explain how to obtain the tenant ID?
There was a problem hiding this comment.
Also, we need to use / endpoint, not /generate.
There was a problem hiding this comment.
updated with information on the tenant short code. I'll need to update the VS to add / as a supported route
| @@ -155,7 +155,9 @@ def generate( | |||
| payload = resp.json() | |||
| if resp.status_code != 200: | |||
| raise parse_error(resp.status_code, payload) | |||
| return Response(**payload[0]) | |||
| if isinstance(payload, list): | |||
| return Response(**payload[0]) | |||
There was a problem hiding this comment.
This smells like bad design IMO. The REST API should be consistent and predictable. We should not be writing client-side hacks to account for inconsistency in the REST API.
No description provided.