simplify neovim plugin addition

[teto]

Issue description

The way to package neovim plugins (ie, lua-based) is a bit obscure, and this for historical reasons.

=== some history ===
Neovim plugins are packaged with the buildNeovimPlugin function while vim plugins are packaged with buildVimPlugin.
buildVimPlugin is simple: unpack folder + generate doc.
buildNeovimPlugin is more complex. buildNeovimPlugin takes a lua package (by name currently), and reinstalls it via luarocks with a different luarocks config that does a "flat" install (by default luarocks install files in different lib/ and share/ subfolders. As neovim reads from the plugin folder in runtimepath, a "flat" install installs everything in a single folder).

When I introduced neovim plugins as derivatives of lua packages (with the goal of automatic dependencies), many things were brittle in the lua ecosystem (broken updater, bugs in LUA_PATH generation etc) and uploading rockspecs to luarocks.org was a lot of work. So I designed it such that one quickly revert/switch back the lua-based plugins to the traditional vim infrastructure.
=== end ===

The lua-based infrastructure is now robust thanks mostly to the rocks.nvim team so this complexity is unnecessary (and Lazy.nvim and rocks.nvim can now install plugins from luarocks !).
Now is a good time for a cleanup and make it easier to contribute.

Right now to add a neovim plugin, you must:

1. add it as a lua plugin
2. run the lua updater
3. add it as a vim plugin && run the vim
   or 3bis. add a buildNeovimPlugin call to pkgs/applications/editors/vim/plugins/overrides.nix

3/ is redundant with 2. 3bis doesn't look that complex haskell-tools-nvim = neovimUtils.buildNeovimPlugin { luaAttr = "haskell-tools-nvim"; }; but it isn't very discoverable ( a bunch of buildNeovimPlugin calls scattered in overrides.nix).
The vimPlugins is a composition of overlays so we could add an overlay generated from a list of neovim plugin (
pkgs/applications/editors/vim/plugins/neovim-plugin-names ) ? but then how to reconcile it with maintainers/scripts/luarocks-packages.csv, we would have 2 source of truths.
We could have the luarocks updater spit the pkgs/applications/editors/vim/plugins/neovim-plugin-names. The updater would process neovim luarocks packages differently from other luarocks packages.
We could encode the fact that the package is a neovim plugin either into maintainers/scripts/luarocks-packages.csv or have luarocks(2nix) tell us (if the rockspec contains a "neovim" label for instance).

Sorry for the wall of text, I dont have any proposition I am satisfied with so looking forward for ideas.

cc @GaetanLepage @mrcjkb

[GaetanLepage]

Thank you for laying this down @teto !
Unfortunately, I am currently busy and do not have a lot of time to spend on Nix.
I will tag my fellow nixvim co-maintainers @MattSturgeon and @khaneliman in case they have further feedback.

[mrcjkb]

I like the idea of a simple neovim-plugin-names file (or neovim-luarocks-plugin-names).
Since neovim plugins should all have the same constraints (lua version = 5.1 or luajit, ...), we could probably remove the nvim plugin entries in luarocks-packages.csv.

[teto]

the luarocks updater (like the vim plugins updater) can accept different input and output filenames.
So we could extract neovim plugins from luarocks-packages.csv and move them to luarocks-neovim-packages.csv. And as output it would generate an overlay directly loadable into vimPlugins. What I like about this is that it makes it easier to update neovim plugins without having to update the rest of luarocks packages, which trigger mass rebuilds (Ideally we could filter which packages to update but until someone fixes it or pays me to do it, we wont have it for a while :'()

[khaneliman]

Sorry, I'm not the most familiar with the lua infrastructure in nixpkgs. I wasn't aware that it was expected to update luarock-packages.csv when adding a neovim plugin. I know for sure the luarocks-packages.csv and vim-plugin-names lists are out of sync just because I've added new neovim plugins that won't be in that csv.

I would definitely encourage moving the neovim plugins from luarocks-packages.csv into their own file to make it more clear and if that reduces rebuilds, that's a big win. Is there a preference to using luarocks over github / github over luarocks sources for plugins or would it be expected to maintain entries for both?

[PerchunPak]

How is building a Neovim plugin using buildVimPlugin different from building it with buildNeovimPlugin?

I wasn't aware that it was expected to update luarock-packages.csv when adding a neovim plugin.

@khaneliman, don't worry, there is a ton of Neovim plugins in vim-plugin-names

[mrcjkb]

@PerchunPak

How is building a Neovim plugin using buildVimPlugin different from building it with buildNeovimPlugin?

There are many Neovim plugins which exist as luarocks packages; which has the benefit that it's the plugins, not the users, who declare dependencies and build instructions.

buildNeovimPlugin lets you reuse the luarocks package, including its dependencies;
buildVimPlugin doesn't.

[teto]

@PerchunPak I would like more documentation for neovim + nixpkgs since it's clearly a recurrent topic. Feel free to improve the doc (I see you wanna help the vim ecosystem and you are so fast, we dont want you to run out of ideas :p). Neovim + treesitter is a recurrent question for instance xD

[teto]

I've put up an ugly PR #342884 but which works. I dont really like that it becomes a separate updater but for the time being, it might in fact be simpler :s I will try to do some prior refactoring before completing that PR.

[khaneliman]

I've put up an ugly PR #342884 but which works. I dont really like that it becomes a separate updater but for the time being, it might in fact be simpler :s I will try to do some prior refactoring before completing that PR.

Is that a separate approach to #336137 or in addition to?

[PerchunPak]

I've put up an ugly PR #342884 but which works. I dont really like that it becomes a separate updater but for the time being, it might in fact be simpler :s I will try to do some prior refactoring before completing that PR.

Is that a separate approach to #336137 or in addition to?

From the first glance, #342884 solves problems specifically for Neovim plugins that are packaged as luarocks, while #336137 solves entirely different set of problems with Vim plugins updater (it almost doesn't touch lua updater). Though it is just from the first look, I don't have much time rn to read the code more