Setting a default chosen endpoint

[paychex-ssmithrand]

Hello,

We are looking to set Azure as the default endpoint selected. Looking through the code it's not completely clear to me where the source of the default setting of "openAI" is.

[heathriel]

Azure pops up as the top one in the list. if you have it set. You can also comment out everything else if you only want that.

[paychex-ssmithrand]

Thanks for the reply! Would you mind pointing out where exactly we should be commenting out? I've tried doing that in a few different places but without success, unfortunately.

The behavior we are seeing is that when LibreChat detects AzureOpenAI as being available, a user still needs to open the menu and change from "openAI" to "AzureOpenAI".

[jrkropp]

I'm seeing the same thing as paychex-ssmithrand. Even though OpenAI is disabled, the user needs to manually go in and change the endpoint to Azure Open AI. It would be nice to set the default endpoint.

[danny-avila]

api/server/controllers/EndpointController.js

comment out what you don't need

I'm seeing the same thing as paychex-ssmithrand. Even though OpenAI is disabled, the user needs to manually go in and change the endpoint to Azure Open AI. It would be nice to set the default endpoint.

To prevent this, I'll try to reproduce, so you don't have to edit the code

[cbutler-captech]

Is there already a planned fix for this? This is a major issue with launch internally with Azure only. My team member has been looking at this and could potentially submit a pr with an optional variable that sets the default endpoint, where if it is blank nothing changes.

[danny-avila]

@cbutler-captech can you describe the issue in detail so i fully understand?

This wouldn't really be a fix in my book rather a feature. Here's what I have planned and you can let me know if it will work for you:
I want to make an environment variable for each endpoint, specifically for disabling it:

DISABLE__OPENAI=TRUE
DISABLE__BINGAI=TRUE
# etc.

That way, you could disable each endpoint, except azure; therefore all new chats and such will always be azure. This is probably the easiest, best thing to do. Will that work?

[jrkropp]

Yes that would work. Although it would be more ideal if you could set the default endpoint. (E.g. we have Azure Open AI enabled and prefer it over OpenAI although we also have OpenAI enabled for failover / new models not released on azure).

The issue is if a new user logs in it defaults to OpenAI even if you don’t have the OpenAI api enabled. The user will receive an error if they try and chat. They need to click on the endpoints and select Azure OpenAI (the only endpoint in the list). Right now we are instructing users to manually select it buts it’s not intuitive.

[cbutler-captech]

I agree with the default endpoint approach, potentially solves 2 problems.

We have disabled all enpoints and variables around everything but Azure and Plugins, with Plugins set to azure. When a new user logs in, they see the Green OpenAI icon and if they type a question, it fails (goes to the openAI endpoint with no key). The issue is that the default conversation somehow is tied to OpenAI, even though it is disabled. Tracking this down is difficult, but there is a bug somewhere in that stack, but the above approach would solve it most likley.

[fuegovic]

I'm thinking out loud here, could a mix of both idea be implemented, something like the model list but for endpoints.

ENDPOINTS=azure,openai,bing,anthropic,...

with the first in the list being the default, and endpoints not in the list being disabled

🤔

[cbutler-captech]

This is also a good approach but the bug is somewhere with the default conversation and would most likely still appear. Perhaps someone more familiar can find it, I will ask my colleague to chime in, but it has to do with the default conversation endpoint being null.

[danny-avila]

The endpoints are listed in the order given here: client/src/utils/getDefaultEndpoint.ts

When I reorder it with Azure first and no local storage saved, like this:

export const defaultEndpoints: EModelEndpoint[] = [
  EModelEndpoint.azureOpenAI,
// ...rest omitted

I get Azure first:

I may just order Azure first by default since needing it first is usually the case for azure deployments.

I mention local Storage because your last conversation setup determines what the endpoint is on load/new chat.

@jrkropp I can't reproduce what you're seeing with the user getting OpenAI selected even when it's disabled with the above change.

I am seeing it without the change, but I've identified what is causing this and will get a fix in soon. For now, I'll change the order of the "defaultEndpoints"

Part of the problem with configuring options for the frontend is that they are retrieved from a network request. The app needs default values even before the network request. So a configuration here may need to be done in a separate file from the .env since it can't read this for security reasons.

[danny-avila]

@cbutler-captech hopefully I can take this off your plate and will fix soon.

[danny-avila]

I have a fix already I'm testing

[danny-avila]

Fix will go live shortly @fuegovic @paychex-ssmithrand @jrkropp @cbutler-captech @JTimmeCaptech

#1206