What is the purpose of /api/conversation/azure_byod in the chat app?

[andrewwiebe]

In the backend of the chat app there is a route /api/conversation/azure_byod. It seems like it is never called from the front end, only /api/conversation/custom is ever called.

I am trying to add a filter parameter on the body that is used in the prepare_body_headers_with_data function like below:

"dataSources": [
            {
                "type": "AzureCognitiveSearch",
                "parameters": {
                    "endpoint": AZURE_SEARCH_SERVICE,
                    "key": AZURE_SEARCH_KEY,
                    "indexName": AZURE_SEARCH_INDEX,
                   "filter": "search.in(fieldName, 'value')",

Link to code

However is seems like that body is only used on the azure_byod route which is never called from the front end.

Can anyone help me with a way to add a parameter onto the body that is sent to the openAI instance?

[david-1m]

I have researched this repository for weeks now and I think it just gives you three methods that you can choose from yourself. Indeed, azure_byod seems to never be called. I am rerouting api/conversation/custom to azure byod and it works amazing.

[andrewwiebe]

Thanks, yeah we have been working on our own modifications on this repo. I discovered you can just change the api call in the front end of the chat app to use the byod route and it seems to working just fine. That is what we are doing for now. Thanks for the note.

[david-1m]

Have you been able to make sense of all the utilities in this repo? To my understanding, they can be used to implement your own logic. I would love to use Azure BYOD but I have multiple indexes and their retrieval approach also isn't the best for our documents. I looked into the utilities and it seems that they add the sources retrieved from AI search as a "tool" message before they send it to the frontend. However, this doesn't work with a streaming LLM, since they haven't implemented the custom callback function yet. My approach now is to make something similar to azure BYOD and add a tool message in the same fashion that is then read out by the frontend. It seems to me like this repository is unnecessarily complicated with all the different utilities.

[adamdougal]

Heya!

My understanding is that the /api/conversation/azure_byod endpoint should be used if the default response and configuration of Azure OpenAI meets your requirements.

If you need to do any pre or post processing, e.g. using langchain or OpenAI functions, then /api/conversation/custom should be used.