Linking chat prompt with trace

[faileon]

Hello all,
the following code:

            const langfusePrompt = await this.langfuse.getPrompt(name, version, { label: 'production', type: 'chat' });
            const prompt = ChatPromptTemplate.fromMessages(langfusePrompt.getLangchainPrompt().map((m) => [m.role, m.content])).withConfig({
                metadata: {
                    langfusePrompt,
                },
            });

            return prompt.pipe(llm.withStructuredOutput(answerSchema)).stream(
                {
                    currentDate,
                    question: input.question,
                    context,
                },
                { callbacks: [langfuseHandler] }
            );

is not linking the prompt with the trace correctly. Only when I add and empty pass through to the chain:

            const test = async (prompt) => {
                console.log('???', prompt);
                return prompt;
            };

            return prompt
                .pipe(test) // or new RunnablePassthrough() from langchain
                .pipe(llm.withStructuredOutput(answerSchema))
                .stream(
                    {
                        currentDate,
                        question: input.question,
                        context,
                    },
                    { callbacks: [langfuseHandler] }
                );

It starts working and my trace is correctly linked to the prompt. Could somebody please help me explain why and who is it to blame?

[dosubot[bot]]

I couldn't find specific information about the internal mechanisms of the langfuseHandler callback or the RunnablePassthrough that might explain why adding a passthrough function resolves the issue. However, it seems that introducing a passthrough function or RunnablePassthrough in the chain affects the data flow in a way that allows the prompt to be correctly linked with the trace.

A similar unsolved discussion might be relevant: Cannot connect prompt to trace when using Langchain AzureChatOpenai, which was last updated on 2024-07-19 ^[1]. This might provide additional insights or context related to your issue.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[marcklingen]

This is interesting, can you open a bug issue as this is an interesting case to investigate and improve

[faileon]

Done #5162 (comment)

I will try to create reproducible repository when I get my hands free