[BUG] Azure GPT 4o is limited to maximum 4096 output tokens even though model supports 16,384

[tsoernes]

Description

Cannot set it higher than 4096 even though model supports it.

Device and browser

Linux

Screenshots and more

No response

Willingness to Contribute

• 🙋‍♂️ Yes, I would like to contribute a fix.

[enricoros]

@tsoernes confirmed. Since the APIs don't tell us the token window and completion sizes, we have to have mappings in the code inside Big-AGI, and the model ID for azure is probably not available either (you have probably the deployment name).

Probably in the future we'd need to make the value fully user editable, or finally OpenAI/Azure will have descriptive APIs where they return the token sizes.

To fix this now, you need to go in the code, models.ts, and look for OpenAI / Azure.