langgraph with local llm

[cesi08]

I'm trying to use the example of the supervisor (https://github.com/langchain-ai/langgraph/blob/main/examples/multi_agent/agent_supervisor.ipynb )made with langgraph with a mistral model deployed locally

the llm definition in my case is

from langchain_openai import ChatOpenAI
llm = ChatOpenAI(base_url="http://localhost:1234/v1", api_key="not-needed")

and I have the following exception
OutputParserException: Could not parse function call: 'function_call'

I think it's due to the way the supervisor chain is defined
​

any suggestion? @hinthornw

[MoMoWongHK]

push, same problem

[hodgesz]

This could also be due to Mistral's less robust function calling support than the paid models. It doesn't support the new tool API from OpenAI at all and only limited function calling support unless fine-tuned is my understanding.

[hinthornw]

Those examples in that notebook use function calling / tool calling, so your API would have to support it if you wanted to apply it directly: https://python.langchain.com/docs/use_cases/tool_use/quickstart/#toolfunction-calling

[anshumankmr]

So is there a way to resolve this?

[hinthornw]

The way around this is either to use a local LLM interface that supports tool calling or to modify the agent orchestration code in the doc to use a custom output parser

[ramankhurana]

@anshumankmr I am here to check if you manage to find a solution for this problem? I am at same problem now. It seems the solution still does not exist in the LangChain/LangGraph. Any suggestions/e.g. code will be highly appreciated.

[anshumankmr]

Was not able to find one. Try autogen instead .

[ramankhurana]

Thanks @anshumankmr Could you share an example or perhaps a reference from Autogen that aligns with a supervisor case?