[suggestion] Ask for additional buffer context via user input

[MrGrinst]

There are cases where it would be nice for the LLM to have the context of another file I have open. I don't have a good idea yet on the implementation specifics but I'm wondering if you would be open to a PR that prompts the user for additional buffers to include as context for the LLM.

[frankroeder]

I am open to have something experimental on a different branch, guessing it would take a little bit more time to test this out and search for the best solution.

[eterps]

you would be open to a PR that prompts the user for additional buffers to include as context for the LLM

You might want to check aider for inspiration which does exactly that.
Unfortunately it's a separate TUI app, not a neovim plugin (which is the reason I'm not using it more often). Still a lot can be learned from it.

[MrGrinst]

@frankroeder @eterps what do you think about having a template variable like {{userinput_buffers}}. When that var is detected in the template, it would use fzf-lua to ask the user to select from open buffers.

[frankroeder]

@MrGrinst this is what I already have on my local branch. Currently, just don't have the time to finish it.

[frankroeder]

@MrGrinst, how is it going with this feature? I guess the async stuff of fzf-lua is not so easy to handle.

[MrGrinst]

Yeah I haven't had time to finish it for that exact reason. The async stuff is beyond my Lua skills.

[eterps]

Last week I played around with this nvim integration for aider.

My first impression is that aider doesn't integrate very well in nvim, but I liked the feature of this plugin to automatically add all open buffers as a context.

It really makes sense as long as you have a set of files opened in buffers for the task at hand (otherwise you would need to delete some buffers first).

[frankroeder]

I have added the feature to include the content of all open buffers. Feel free to check out the branch for pull request #37.

Just to be clear, tools like Aider and Devin aim to "replace" the programmer's work. We focus on supporting the programmer through LLMs in the first place.

[eterps]

I have added the feature to include the content of all open buffers.

Oooh nice, that looks incredibly useful. Definitely gonna check it out 👍

We focus on supporting the programmer through LLMs

Yeah, I agree, no need to turn this into Aider/Devin. The way parrot.nvim's templates work offer more fine grained control anyway.

[eterps]

@frankroeder , it works great. Very useful feature.

I noticed that I had to change the last part of the prompt to:

Please finish the code above carefully and logically.
Respond with the snippet of code that should be inserted, but absolutely no explanations or code block markers.

Otherwise Claude 3.5 would keep adding explanations or code blocks.

I'm wondering if this would be nice to have this for chats as well (instead of just for commands).

[frankroeder]

Great, I'm glad to hear that!
Sure, I guess each model needs system prompt fine-tuning at some point, and it is not easy to provide general prompts that work equally well.

I believe this is what we already have with #27 for chats and the general templates in the config that can be overridden for the commands. If I've misunderstood you, could you provide more details on what exactly you want to change and under which circumstances?

[MrGrinst]

This is great, I like this approach! Thanks @frankroeder. I'm thinking of adding a check to skip files larger than a certain size. Wanna avoid expensive calls (already had one go through that was 100k tokens 😮). Would you be open to that?

[MrGrinst]

Well it ended up being pretty hacky and probably making some assumptions based on my usage that might not be true for others. Not gonna open a PR but for reference here's the approach: 2b644d8

[frankroeder]

@MrGrinst, that is a very good suggestion. I am considering adding an optional confirmation for a specific approximate token limit. Thank you very much for your idea and implementation.