OpenAI timeout when using Llama models

[epistoteles]

The API is under heavy load right now. I'm trying to work on some Llama models.

However, it seems the API is doing some OpenAI requests in the background? At least I'm getting a 60 second OpenAI timeout error message.

Could you look into this and potentially remove OpenAI requests from Llama chats? This might free up some capacity.

[dedeswim]

From the Google Group announcement from yesterday:

Our TogetherAI and OpenAI accounts have rate limits; there’s nothing we can do about this. If you start getting OpenAI errors (this is the error it will raise for both models, as we’re indirectly using openai-python for both) using the team budget tomorrow, try using your own keys.

[epistoteles]

Alright, thanks for the quick response!

[dpaleka]

It seems TogetherAI upstream API is having issues. Can some participants try if it's also extremely slow for them too, on their personal keys?

If this is the root cause, there is really nothing we can do about this, except decrease \beta. We'll do an announcement shortly.

[s-zanella]

I confirm that Together API is also unresponsive using paid tier keys.

[dedeswim]

We released an announcement. Please rereport any errors that are not covered by the announcement, thanks!

[dedeswim]

Looks like their API is up again

[dedeswim]

Looks like Together's API is having some issues again. We'll monitor the situation and act if the problem does not get solved soon.

[dedeswim]

Looks like the API is up again, closing but keeping pinned.

[Sandr0x00]

lol together ... of course 2h before the end :)

[dedeswim]

It seems to be up again?

[Sandr0x00]

Yep, only was down like 20-30min or so, luckily got up again :)