Visualisation of graph execution

[VMinB12]

Describe the feature or potential improvement

First off, thank you for creating this open source tool. It's really wonderful and excellent.

One feature that I think will be really nice and would allow Langfuse to distinguish itself from competitors, is to add better support for graph visualisation. Most LLM applications can be represented as a graph. And the execution flow can be considered as traversing through the graph.
Graph-based agents are all the rage/hype these days and I expect it will become even more so in the future. Providing this feature sooner rather than later may significantly help adoption of this platform for those use cases.

Some competitors are:

• https://github.com/AgentOps-AI/agentops
• https://langchain-ai.github.io/langgraph/cloud/

I haven't used agentops, but they don't seem to have the graph visualisation yet. I'm sure they'll add it soon enough. Langgraph Cloud is a great example of visualizing the graph and the flow of execution.

I intentionally let the exact visualization a bit vague as to allow for sufficient creativity in the implementation. For a more concrete idea of what I would want myself, see this Langgraph Cloud example. Triggering the graph execution from the Langfuse UI should be out of scope imo, but ideally there would be a "playback slider" allowing the user to slide through the execution flow of graph traces.

Interested to know your thoughts!

Additional information

No response

[marcklingen]

Hi @VMinB12, thanks for raising this! I have chatted with many teams building complex stateful agents and think this generalizes well to a more state-based system where the state of the application can change throughout the trace.

Introduce state: string

Options to adapt the langfuse data model to enable this:

• Add state to observations as an attribute
• Use the name of an observation as the state of the system -> if multiple have the same name, we know that it is circular
• Add a new object type that denotes state changes independent of the current tracing logic

Open questions: Is there a single "global" state within a single agent execution or can there by multiple states at different levels. I think we need the latter here to support multiple nested levels of agent state. As observations are nested in langfuse, we could visualize state changes on each grouping level and default to the global/trace-level.

Would love to hear your thoughts, this is not high yet on the very near-term roadmap but something I am very excited about and would love to push.

[VMinB12]

Hi @marcklingen, thanks for considering this.
Regarding your open questions:
In LangGraph there is a single "global" state that is updated with every "superstep" (LangGraph is inspired by Pregel from which it inherited this notion of superstep).
What do you mean with "level" here?

In the meantime, LangChain has launched LangGraph Studio which could be a great source of inspiration.

[david1542]

I also think creating a LangGraph Studio experience inside Langfuse would be great!

[AlithAnar]

Hi! We are creating graphs of multiple nodes some of them are sub-graphs with the separate state (they are aggregating data in parallel runs and then share some state to the main graph state).

What we expect from the graph visualization in LangFuse is a mental model view, where a non-technical person (like the support team member) could trace step-by-step the flow and easily discover which next node (agent) LLM chose (and why). Think of it like a supervisor architecture presented by the LangChain team:

Nodes communicates with each other so it would be super-helpful to highlight the communication steps on the edge between nodes (e.g. last the message / command sent from the supervisor node to the expert_1 node).

Example graph:

[marcklingen]

Thanks so much for talking about this on Discord and sharing this here!