Skip to content

Commit

Permalink
Refactor model availability check in is_serverless_endpoint_available (
Browse files Browse the repository at this point in the history
…#363)

* Refactor model availability check in is_serverless_endpoint_available() function

* Refactor model ID retrieval in is_serverless_endpoint_available() function
  • Loading branch information
davanstrien authored Mar 1, 2024
1 parent f255c16 commit f9e41b6
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions src/distilabel/llm/huggingface/inference_endpoints.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,11 +66,11 @@ def is_serverless_endpoint_available(model_id: str) -> bool:
# 2. Then we check if the model is currently deployed
try:
client = InferenceClient()
deploy_llms = client.list_deployed_models("text-generation-inference")[
"text-generation"
]
if model_id in deploy_llms:
return True
status = client.get_model_status(model_id)
return (
status.state in {"Loadable", "Loaded"}
and status.framework == "text-generation-inference"
)
except Exception as e:
logger.error(e)

Expand Down

0 comments on commit f9e41b6

Please sign in to comment.