Python SDK - get or create prompt

[SupraSummus]

Describe the feature or potential improvement

It would be great for my use-case to transparently "get or create prompt". Something similar to gettext translations. Or maybe there already is something like this implemented?

For example I would call

lf_prompt = langfuse.get_or_create_prompt(
    name="movie-critic",
    prompt="As a {{criticlevel}} movie critic, do you like {{movie}}?",
    ...
)

It would act like a get_prompt with fallback parameter, but would create langfuse prompt (in the cloud) on miss. On hit it would return cloud-managed version and ignore prompt=... parameter. On 401, network failure or other it would return prompt and would not create anything, because there is no way to do it.

And there is a slight chance that my idea is a bad design pattern ;) Cheers

Additional information

No response

[SupraSummus]

Hmm, or maybe dont create missing prompts, but store this info in the trace. Then in the CMS user could select "create prompt version from this generation".

Currently it seems that trace-prompt link is not created when fallback prompt is used (from https://langfuse.com/docs/prompts/get-started#link-with-langfuse-tracing-optional).

[marcklingen]

That's correct, the fallback is completely local and was added at a time where the caching was not as robust. Today, I'd recommend to use prefetching if necessary but otherwise rely on Langfuse for prompts as they are highly available

[marcklingen]

Thanks for sharing this!

Have you had a look into fallback yet? I think creating a prompt in langfuse on each use can be tricky as it is unclear what kind of effect this has on versioning and caching - like do we need to run a check every time the prompt is used whether the current version in code does not correspond to the version in Langfuse?

[SupraSummus]

Let me restate and focus more on the issue I have, and less on the solution I propose.

When developer adds a new prompt in the code, lets say it is a totally new process with new data and so on, they expect 100% control over what is happening in the code. The new code should work on the local development environment without the need to connect to langfuse. On CI there are some tests that trigger the new prompt, and it happens without communication with langfuse also. And even if we would assume communication with langfuse at this early development stages, then adding new functionality would require working in two very separate places: the code and langfuse cms. This is a bit cumbersome -- "cumbersome" boils down here to violating locality of behavior. And there is some preference aspect to this -- as for me, the programmer, I love configuring things in code and I hate configuring things in CMSes, or more generally, by clicking in application-specific interfaces.

(This things above is nicely mitigated by fallback.)

The second part is about using the CMS, which of course have its advantages. I'm focusing on exposing prompt configuration to non-technical staff. This is my core motivation for using langfuse. Non-technical developers need a starting point for modifying the prompt. I feel like I cant expect them to create a new prompt with correct parameters and name and type, etc. I anticipate a lot of mistakes and frustration here (when you need to run an 10-minute long pipeline to test if your prompt is picked from CMS). And even skipping the error-prone nature of writing prompt from scratch in the CMS, I feel like it is extra work that can be avoided. Programmer already provided the "basic" prompt in the code, and it could be automatically transferred to CMS.

Okay, so maybe programmers need to create a machinery for creating basic prompt versions in CMS based on code? Python SDK have required functions for it (we can test if prompt exists in the CMS and we can create new prompt). I think some form of this machinery should be available out of the box in the SDK, and should require no setup. Okay, maybe minimal setup.

Now the solution: I argue that modifying get_prompt to insert fallback into CMS when prompt does not exist is nice solution.

Side thought: When the prompt in the code changes in meaningful way, lets say it have different placeholders, then maybe SDK should create new version? Or manage this change somehow, because the CMS-provided prompt version becomes not compatible with the code. This may be resolved by code developers being very careful to change the name of the prompt when signature changes, but can we expect devs to be careful? Well, don't expect me to be careful ;)

@marcklingen what do you think? Is this use case coherent with langfuse general use intention? Maybe there are functions that solve this issues, but I just haven found them or understood correctly? Or maybe there is another, better process to managing prompts?