Use permchain agent executor, Streaming, API docs #45
Conversation
nfcampos
commented
Nov 14, 2023
•
nfcampos
commented
- Figure out streaming output
|
if yall get streaming, people will use this over Assistants imo |
| def _create_function_message( | ||
| agent_action: AgentAction, observation: str | ||
| ) -> FunctionMessage: | ||
| if not isinstance(observation, str): |
There was a problem hiding this comment.
can't see from current code but when is observation not str? maybe update type signature?
There was a problem hiding this comment.
its often not (sadly)
type hints need updating
| # TODO if this is last step, return stop message instead | ||
| return tool_chain | ||
| elif isinstance(message.additional_kwargs.get("agent"), AgentFinish): | ||
| return RunnablePassthrough() |
There was a problem hiding this comment.
Confirming -- this stops execution because there was no value update? How does it check for updates under the hood? Does it compare serialized representations? Is RunnablePassthrough() special?
There was a problem hiding this comment.
It stops execution because it doesn't write to any channel (execution continues only when channels are written to)
|
|
||
| def get_agent_executor( | ||
| tools: list[BaseTool], | ||
| agent: Runnable[dict[str, list[AnyMessage]], AgentAction | AgentFinish], |
There was a problem hiding this comment.
Hm at some point we could add another executor to support also agents that return a list of AnyMessage (will help remove the boiler plate for working with something like openai directly)
| @@ -33,7 +33,6 @@ | |||
| [ | |||
| ("system", template), | |||
| MessagesPlaceholder(variable_name="messages"), | |||
| ("ai", "{agent_scratchpad}"), | |||
| ] | |||
| channels={"messages": Topic(AnyMessage, accumulate=True)}, | ||
| input=["messages"], | ||
| output=["messages"], | ||
| checkpoint=checkpoint, |
There was a problem hiding this comment.
When are values written to the checkpoint? And what is the content of checkpoint? Is a checkpoint taking place after each update iteration is completed? Is the content of checkpoint all the messages so far or just the new messages? (Seems like the former)
There was a problem hiding this comment.
Check the implementation in permchain, we have a choice of values being written to checkpoint storage either at the end of each step, or once at the end of the run. By default is end of each step
| else: | ||
| return AIMessage( | ||
| content=output.return_values["output"], | ||
| additional_kwargs={"agent": output}, |
There was a problem hiding this comment.
what happens to these kwargs when we send them to the llm? do they get stripped out?
There was a problem hiding this comment.
Our ChatModels dont use additional kwargs they dont know about
|
|
||
| return Pregel( | ||
| chains={"executor": executor}, | ||
| channels={"messages": Topic(AnyMessage, accumulate=True)}, |
There was a problem hiding this comment.
How does accumulation work?
There was a problem hiding this comment.
Add partial validation (first pass)
WIP Stream message tokens
nfcampos
changed the title
