Inconsistent results with LangChain

[salvadr]

Hi, I'm trying to add traces to our LangChain/LangGraph code and I can't manage to get consistent results, specifically in relation to tags and sessions. I've tried 2 approaches:

1/ With CallbackHandler:

I'm using this code copied from the docs:

from langfuse.callback import CallbackHandler
langfuse_handler = CallbackHandler()
trace_name = "langchain_trace_name"
session_id = "random-session"
user_id = "random-user"
tags = ["random-tag-1", "random-tag-2"]

[...]

llm.invoke(
            prompt,
            config={
                "callbacks": [langfuse_handler],
                "run_name": trace_name,
                "tags": tags,
                "metadata": {
                    "langfuse_session_id": session_id,
                    "langfuse_user_id": user_id,
                })

With this I don't get the tags nor the session (the tags are present in the "metadata" area), but I do get the input and output.

2. With trace:

langfuse = Langfuse()
langfuse_trace = langfuse.trace(name="job.analyst_agent", tags=["job", "analyst_agent"],  session_id="ABCDE")
langfuse_handler = langfuse_trace.get_langchain_handler()

[...]

llm.invoke(
            prompt,
            config={
                "callbacks": [langfuse_handler]
            })

With this I do get the tags and session parsed correctly but the input and output are null. And I get an empty duplicate record of the trace.

I'd favor the first approach, so that I can set the session id with the metadata.

Any ideas are welcome, thanks in advance.

[dosubot[bot]]

Hello, @salvadr! I'm here to assist you with any issues you're experiencing. If you have any questions or need help with bug fixes, feel free to ask.

To achieve consistent results with tags and sessions in LangChain/LangGraph using the CallbackHandler approach, while ensuring that the input and output are correctly captured, you can follow these steps:

1. Initialize the CallbackHandler with Tags and Session Information:
   You can pass tags, session ID, and user ID directly to the CallbackHandler constructor to ensure they are consistently applied to all traces handled by this instance.

   from langfuse.callback import CallbackHandler
   
   handler = CallbackHandler(
       tags=["random-tag-1", "random-tag-2"],
       session_id="random-session",
       user_id="random-user"
   )

2. Invoke the Chain with Configurations:
   When invoking your chain, pass the CallbackHandler instance and any additional tags through the configuration. This ensures that the input and output are captured with the specified tags and session information.

   llm.invoke(
       prompt,
       config={
           "callbacks": [handler],
           "tags": ["random-tag-1", "random-tag-2"],
           "metadata": {
               "langfuse_session_id": "random-session",
               "langfuse_user_id": "random-user",
           },
       },
   )

3. Use the @observe() Decorator for Dynamic Tagging:
   If you need to set tags dynamically, you can use the @observe() decorator and update the current trace with tags using langfuse_context.

   from langfuse.decorators import langfuse_context, observe
   
   @observe()
   def fn():
       langfuse_context.update_current_trace(
           tags=["random-tag-1", "random-tag-2"],
           session_id="dynamic-session",
           user_id="dynamic-user"
       )
       langfuse_handler = langfuse_context.get_current_langchain_handler()
       llm.invoke(prompt, config={"callbacks": [langfuse_handler]})
   
   fn()

By following these steps, you ensure that the tags and session information are consistently applied to your traces, and the input and output are correctly captured using the CallbackHandler approach. Additionally, using the @observe() decorator allows for more dynamic and flexible tagging within your functions ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[hassiebp]

@salvadr Could you please try wrapping your LLM execution in a chain / Runnable? The config you have passed is only supported on chains and not if using a bare LLM.