[WIP] Configurable prompt templates

[dlqqq]

TODO: description. Fairly self-explanatory though.

[krassowski]

I see an appeal for making this into a traitlet, but wanted to ask what I believe is an important question: should it be customizable on per-provider or per-model basis instead? It feels orthogonal to the existing get_prompt_template prompt customisation used for %ai magics.

Also linking #498.

[krassowski]

I am happy to open a counter PR with the per-provider strategy if that makes sense.

[dlqqq]

@krassowski Agreed. I think that doing this on a per-model basis makes sense.

I am happy to open a counter PR with the per-provider strategy if that makes sense.

I'm currently encouraging other contributors to use providers as little as possible in their APIs.

I've been thinking a lot about the need for a new major release of Jupyter AI, one that invalidates the earlier design decisions that I now regret. One of the biggest pain points has been the notion of providers. Providers are an implementation detail that should be abstracted away; they are extremely prevalent in our code but are found nowhere user-side. It's so confusing how providers are only providers when they're classes, but become more like "models" when they're instances. Finally, it creates a ton of ambiguity on how to implement model-specific configuration.

Unfortunately, this year, I kept running out of time to pursue these ideas further. In 2024, hopefully I will have time to implement this.

[krassowski]

One of the biggest pain points has been the notion of providers. Providers are an implementation detail that should be abstracted away; they are extremely prevalent in our code but are found nowhere user-side.

I think I know some of the pain points you are referring too ;) Is there an issue discussing these plans? That would allow users and developers to express their thoughts and maybe offer some helpful suggestions. I can see argument for going either way, and given the project prominence I think it would be hugely beneficial to the community if we had an opportunity to discuss major design decisions in the open.

PS. even if you do not have a plan to describe the detailed proposal now, just having a placeholder issue with one sentence to refer to would be helpful :)

[dlqqq]

@krassowski Feel free to open an issue with your thoughts! Just reference the discussion here for context. Then, when I plan the next major release of Jupyter AI (what I'm calling "Jupyter AI Next" for now), I will incorporate your suggestions.