Something that could actually help my company (dynamic system message) #130
giladbarnea
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi!
The company I work at is rolling out a llm-based chatbot.
We've been using the
chat
feature and it's great! You could remove theexperimental
tag as far as I'm concerned.One major pain that comes up over and over, especially among product managers, is the inability to know all possible edge cases and user flows. This is inherent to LLMs but it's a tough paradigm shift for product and design folks.
One potential solution is to tweak the system message as real world feedback comes in. However, this often leads to overfitting.
Another is to use agents, but these are slow and expensive (ReAct and the likes).
But I think you could get the best of both worlds if you set up a fast and cheap agent that would update the system message on-the-fly, completely tailored to the current conversation.
This agent's inputs would be the last user message and the current system message. It would be asked what's the user message about, and then asked to incorporate it into the system message. From that point on the conversation is painted differently due to the system message's weight.
Some diagrams always help:
Beta Was this translation helpful? Give feedback.
All reactions