LLMError: Hugging Face request failed: Server error '500 Internal Server Error' for url 'https://api-inference.huggingface.co/models/mistralai/Pixtral-Large-Instruct-2411/v1/chat/completions' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500

[zhimin-z]

When I attempt with mistral model as follows:

import os
import aisuite as ai

# Either set the environment variables or define the parameters below.
# Setting the parameters in ai.Client() will override the environment variable values.
client = ai.Client()

model = "huggingface:mistralai/Pixtral-Large-Instruct-2411"  # Replace with your model's identifier.

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What's the weather like today?"},
]

response = client.chat.completions.create(
    model=model,
    messages=messages,
)

print(response.choices[0].message.content)

It throws the above error:

---------------------------------------------------------------------------
HTTPStatusError                           Traceback (most recent call last)
File ~/Documents/GitHub/SE-Arena/.venv/lib/python3.12/site-packages/aisuite/providers/huggingface_provider.py:50, in HuggingfaceProvider.chat_completions_create(self, model, messages, **kwargs)
     [49](https://file+.vscode-resource.vscode-cdn.net/Users/jimmy/Documents/GitHub/SE-Arena/~/Documents/GitHub/SE-Arena/.venv/lib/python3.12/site-packages/aisuite/providers/huggingface_provider.py:49)     response = httpx.post(url, json=data, headers=headers, timeout=self.timeout)
---> [50](https://file+.vscode-resource.vscode-cdn.net/Users/jimmy/Documents/GitHub/SE-Arena/~/Documents/GitHub/SE-Arena/.venv/lib/python3.12/site-packages/aisuite/providers/huggingface_provider.py:50)     response.raise_for_status()
     [51](https://file+.vscode-resource.vscode-cdn.net/Users/jimmy/Documents/GitHub/SE-Arena/~/Documents/GitHub/SE-Arena/.venv/lib/python3.12/site-packages/aisuite/providers/huggingface_provider.py:51) except httpx.HTTPStatusError as http_err:

File ~/Documents/GitHub/SE-Arena/.venv/lib/python3.12/site-packages/httpx/_models.py:829, in Response.raise_for_status(self)
    [828](https://file+.vscode-resource.vscode-cdn.net/Users/jimmy/Documents/GitHub/SE-Arena/~/Documents/GitHub/SE-Arena/.venv/lib/python3.12/site-packages/httpx/_models.py:828) message = message.format(self, error_type=error_type)
--> [829](https://file+.vscode-resource.vscode-cdn.net/Users/jimmy/Documents/GitHub/SE-Arena/~/Documents/GitHub/SE-Arena/.venv/lib/python3.12/site-packages/httpx/_models.py:829) raise HTTPStatusError(message, request=request, response=self)

HTTPStatusError: Server error '500 Internal Server Error' for url 'https://api-inference.huggingface.co/models/mistralai/Pixtral-Large-Instruct-2411/v1/chat/completions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500

During handling of the above exception, another exception occurred:

LLMError                                  Traceback (most recent call last)
Cell In[4], [line 15](vscode-notebook-cell:?execution_count=4&line=15)
      [8](vscode-notebook-cell:?execution_count=4&line=8) model = "huggingface:mistralai/Pixtral-Large-Instruct-2411"  # Replace with your model's identifier.
     [10](vscode-notebook-cell:?execution_count=4&line=10) messages = [
     [11](vscode-notebook-cell:?execution_count=4&line=11)     {"role": "system", "content": "You are a helpful assistant."},
     [12](vscode-notebook-cell:?execution_count=4&line=12)     {"role": "user", "content": "What's the weather like today?"},
     [13](vscode-notebook-cell:?execution_count=4&line=13) ]
---> [15](vscode-notebook-cell:?execution_count=4&line=15) response = client.chat.completions.create(
     [16](vscode-notebook-cell:?execution_count=4&line=16)     model=model,
...
     [53](https://file+.vscode-resource.vscode-cdn.net/Users/jimmy/Documents/GitHub/SE-Arena/~/Documents/GitHub/SE-Arena/.venv/lib/python3.12/site-packages/aisuite/providers/huggingface_provider.py:53) except Exception as e:
     [54](https://file+.vscode-resource.vscode-cdn.net/Users/jimmy/Documents/GitHub/SE-Arena/~/Documents/GitHub/SE-Arena/.venv/lib/python3.12/site-packages/aisuite/providers/huggingface_provider.py:54)     raise LLMError(f"An error occurred: {e}")

LLMError: Hugging Face request failed: Server error '500 Internal Server Error' for url 'https://api-inference.huggingface.co/models/mistralai/Pixtral-Large-Instruct-2411/v1/chat/completions'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500

[ksolo]

@zhimin-z Is this still an issue for you? 500 errors are normally issues in the providers service, it could have been caused by some type of outage. If this has resolved, please close the issue but if it is still an issue we can look into it.

[zhimin-z]

@zhimin-z Is this still an issue for you? 500 errors are normally issues in the providers service, it could have been caused by some type of outage. If this has resolved, please close the issue but if it is still an issue we can look into it.

Still there yet, does it occur on your side as well?

[ksolo]

@zhimin-z I will take another look at this on later this week.