Langfuse Launch Week #1

[clemra]

We’re excited to announce Langfuse's first launch week. We're kicking it off on Monday April 22nd and will release a major upgrade to the Langfuse platform every day until Friday. We will be updating this post regularly with all news!

• ⭐️ Star us on Github & see all of our releases!
• Twitter will be our main channel for all announcements of Launch Week 1
• Join our first town hall on Wednesday (more below)

Townhall

You're invited to our first virtual town hall. We (Max, Marc and Clemens) will be demoing new features in Langfuse, answering questions and talking about where we're taking the project. We're looking forward to hanging out!

• When: Wednesday, April 24th, noon PT, 9pm CET
• Where: Google Meet
• Sign up here

Launches

Day 0: OpenAI JS SDK Integration

We launched a new wrapper for the OpenAI JS SDK. This integration, designed to enable easier monitoring of OpenAI API usage, features seamless observability with enhancements like automatic tracking of prompts, completions, and API errors, as well as insights into model usage and costs. After a soft launch that gathered user feedback for improvements, the integration is now fully available, complete with comprehensive documentation and an example notebook.

Day 1: PostHog Integration

We teamed up with PostHog (OSS product analytics) to integrate LLM-related product metrics into your existing PostHog dashboards. This integration is now available in public beta on Langfuse cloud. You can configure it within your Langfuse project settings. When activated, Langfuse sends metrics related to traces, generations, and scores to PostHog. You can then build custom dashboards to visualize the data or use the LLM Analytics dashboard template to get started quickly. See docs for more information.

Day 2: Playground

We're excited to introduce the LLM Playground to Langfuse. By making prompt engineering possible directly in Langfuse, we take another step in our mission to build a feature-complete LLM engineering platform that helps you along the full live cycle of your LLM application. With the LLM playground, you can now test and iterate your prompts directly in Langfuse. Either start from scratch or jump into the playground from an existing prompt in your project. See the docs for more details and let us know what you think in the GitHub discussion.

Product Hunt

We will end the week with the launch of Langfuse 2.0 on Product Hunt on Friday, April 26th. After our initial launch last year – which led to a Golden Kitty Award – we are very excited to be back on Product Hunt.

[clemra]

Day 3: Decorator in Python

We're happy to share that the Decorator-based integration for Python now supports all Langfuse features and is the recommended way to use Langfuse in Python.

The decorator makes integrating with Langfuse so much easier. Head over to the Python Decorator docs to learn more. All inputs, outputs, timings are captured automatically, and it works with all other Langfuse integrations (LangChain, LlamaIndex, OpenAI SDK, ...).

To celebrate this milestone, we wrote a blog post on the technical details and created the example notebook shown in the video as it demonstrates what's really cool about the decorator. Let us know what you think in the GitHub discussion, and stay tuned for more updates during Langfuse Launch Week 1 🚀

Thanks again to @lshalon and @ashishghosh for your contributions to this!

[hassiebp]

@secsilm Thanks a lot for your kind words 🙏🏾

From my current knowledge, the problems with multithreading will occur when the threads are spawned (1) through an executor and (2) inside a trace. For example when a @observe-decorated function uses a ThreadPoolExecutor to make concurrent LLM requests. In that case, the context that holds important info on the nesting hierarchy ("we are inside another trace") is not copied over correctly to the child threads. So the created generations will not be linked to the trace and be 'orphaned'. In the UI, you will see a trace missing those generations.

When spawning threads manually with threading.Thread, I would expect the contextvars to be copied over correctly as no executor is used. Feel free to try it out and please report back to us your results. When testing, you can look out whether your traces include the generations from the spawned threads and whether the correct nesting hierarchy is maintained in the trace.

[secsilm]

Thank you for your reply. So far, I have not observed any errors. And we do not have complex nested structures.

[kikwho]

@hassiebp Maybe an elegant solution for the ThreadPoolExecutor problem would be a trace_id parameter in the @observe decorator?

[marcklingen]

@hassiebp Maybe an elegant solution for the ThreadPoolExecutor problem would be a trace_id parameter in the @observe decorator?

agree, tracking this here: #3190

[hassiebp]

@kikwho - thanks again for the hint! We have just released adding a Langfuse parent IDs to the tracing of decorated function executions. We have documented your idea for the ThreadPoolExecutor workaround here as well as the recommended approach.

[clemra]

Day 4: Datasets 2.0 - New editor, metadata on all objects, tables that render i/o, dataset runs on traces, push traces to datasets, and more.

On Day 4 of Launch Week 1, we're happy to share Datasets v2 as we have made significant improvements to the dataset experience in Langfuse. Improvements include a new editor powered by Codemirror, metadata support on all objects, tables that render inputs/outputs side-by-side, the ability to link dataset runs to traces, and the option to create dataset items directly from traces. We've also extended the public API with new endpoints for programmatic management of datasets.

Changelog: https://langfuse.com/changelog/2024-04-25-datasets-v2

[clemra]

Day 5: Model-based Eval Service - automatically run evals on all incoming traces

Run model-based evaluations on traces in Langfuse to scale your evaluation workflows. Start with one of our battletested templates or use your own custom templates.

On the final day of Launch Week 1, we're happy to release the biggest change to Langfuse yet: Model-based Evaluations.

Video Walkthrough: https://www.youtube.com/watch?v=cv8Obogy7Sw

So far, it was easy to measure LLM cost and latency in Langfuse. Quality is based on scores which can be user feedback, manual labeling results, or be ingested by evaluation pipelines that you built yourself using the Langfuse SDKs/API.

Model-based Evaluations in Langfuse make it way easier to continuously evalutate your application on the dimensions you care about. These can be: hallucinations, toxicity, relevance, correctness, conciseness, and so much more. We provide you with some battle-tested templates to get you started, but you can also write your own templates to cover any niche use case that might be exclusive to your application.

Changelog: https://langfuse.com/changelog/2024-04-26-model-based-evaluation
Docs: https://langfuse.com/docs/scores/model-based-evals

note: this feature is currently only available on Langfuse Cloud

[clemra]

It's a wrap!

You can find all information in our blog post.

Read about Langfuse 2.0 and please continue sending feedback our way!