New Github organization for (Neo)Vim + R

[jalvesaq]

Neovim and R have some differences that make it difficult to take full advantage of either editor's features in a plugin supporting both editors simultaneously. So, following the conversation started at #795, I'm opening this discussion to propose the creation of a Github organization to develop and maintain R-related plugins to Vim and Neovim.

Some of the differences between Vim and Neovim are:

• Only in Neovim:

  • Auto-completion with nvim-cmp.

  • Syntax highlight of code blocks in doc/Nvim-R.txt (depend on nvim-treesitter).

  • Syntax highlight of float windows during omni-completion.

  • Syntax highlight of the R output in the built-in terminal (but colorout is an alternative for Vim on Unix).

  • Possibility of mapping <M--> as R_assign_map.

• Only in Vim (or Neovim with the legacy system of syntax highlighting):

  • Syntax highlight of only loaded functions.

These are differences that users can see. For developers, the most striking difference is that in Neovim it's better to write plugins in Lua while Vim now has a new scripting language (Vim9). I no longer use Vim, and I don't want to keep maintaining code that I don't use. It will be much better for me to freeze the support for Vim and keep developing only plugins for Neovim. Hence, I propose to freeze the development of new features in Nvim-R but not abandon current Vim users; I would keep maintaining the code, fixing newly reported bugs, and backporting bug fixes from the new projects for a while.

The first task of the new organization would be to create substitutes for Nvim-R and cmp-nvim-r. The substitute for Nvim-R would:

• Convert VimScript into Lua.

• Make <M--> the default R_assign_map and remove the complicated support for _ as R_assign_map.

• Eliminate:

  • Support for legacy omni-completion.

  • Support for (un)commenting code.

  • Files:

    R/complete.vim
    R/comment.vim
    R/vimrbuffer.vim
    R/vimcom.vim
    R/gui_running.vim
    R/rmd.snippets
    R/r.snippets
    syntax/rdocpreview.vim

  • Support to anything that languageserver or a general plugin can do better.

A substitute of Nvim-R for Vim could also be developed if someone in the organization was interested. The starting point would be easy: simply delete all specific Neovim code from the current Nvim-R.

A new plugin to be developed would be nvim-dap-r.

Are you willing to join this organization? How could you contribute?

[akthe-at]

What aspects of maintenance for Windows are most troublesome for you @jalvesaq ? I am trying to figure out if I should try to contribute with lua,R, or C code that needs to be geared towards nvim-r support on Windows. As a Windows user that enjoys using this package daily I want to figure out if I can be of service so it can continue development instead of being frozen.

[jalvesaq]

Thanks, @akthe-at! There is nothing especially difficult on Windows, but not every code written on Linux works on Windows. Currently, there is no open bug labeled "Windows", but they may appear at any time in existing code or be introduced with new features. If a bug were reported affecting only Windows, you could fix it. This is great!

It will be much harder to keep support for Vim because the Vim maintainers would have to translate Lua into VimLanguage for every new feature and bug fix.

[hongyuanjia]

As a Windows user, I have also used NVim-R for years. It would be a pity if Windows support is dropped in the future. I want to help with testing NVim-R on Windows or contribute in any other way.

[jalvesaq]

@she3o, if I feature freeze Nvim-R, your contribution will not have any consequences. Hence, instead of merging your pull request, I think it's better to create the organization and incorporate your code there.

Who would like to be a member? Does someone want to contribute with ideas or pull requests, but prefer not to be a member of the organization? @she3o, @akthe-at, @hongyuanjia? @jamespeapen, @klmr, @mllg? Who else?

We should probably agree on minimal rules for inviting or accepting new members. We also have to agree on a name, such "R-Vim-toolkit", "R-Nvim-tookit", "R-Nvim-suite", or another. If we do not foresee having a developer for Vim9, it might be more appropriate to use "nvim", than "vim" in the name.

I can immediately feature freeze Nvim-R. Then, the next step can be either immediately starting the Nvim-R substitute (to preserve the history of the transition) or starting a temporary repository to later copy the code to the definitive Nvim-R substitute and delete the temporary repository.

Either way, I can easily delete Vim specific code. But I would like suggestions on what more could be removed from Nvim-R codebase. I already have in mind:

• Simplify the function ReplaceUnderS(), eliminating the option of mapping _ to <-.

• Remove support for commenting code.

• Remove support for legacy omni-completion.

• Try to replace RWarningMsg with rcarriga/nvim-notify.

• Create a Lua directory and an init.lua with at least the setup() function. Initially, the setup() function will only create the global variables.

Anything else would depend on converting existing VimScript to Lua (AI tools can help with this task) or adding new functionalities written in Lua.

We can bring back removed features if they are requested by users in the future.

I don't have a clear roadmap yet, but a possibility is:

• Use Lua to interface with the nvimrserver (@she3o is working on this).

• Improve integration with the languageserver and quarto.nvim.

[klmr]

I’m supportive of the project reorg although I can’t commit to contributing personally: I may be a somewhat advanced user of Vim but I know neither Vim Script nor Lua to any extent.

[she3o]

Cool I just saw this. I will stop pushing to nvimrserver at the moment and look at lua mugration ideas.

[PMassicotte]

Just throwing this (https://github.com/dgkf/debugadapter) for DAP ideas.

[jalvesaq]

Thanks, @PMassicotte !

I created a temporary repository starting the transition from VimScript to Lua. The configuration is already made in Lua, using the typical setup() function, but almost everything else is still in VimScript: tmp-R-Nvim.

[PMassicotte]

Great! I will start looking to convert some vim into lua!

[jalvesaq]

Good! I will take a break from this for a few days at least. So, there is no risk of duplicate effort.

[PMassicotte]

Just for information, I have started to convert all "send code" functions into lua

[jalvesaq]

Great! I will work on other files, but I will have time only on Sunday. An idea: as a transitional measure, to be able to test newly converted functions, if necessary and possible, we could keep a definition of an old VimScript function as a wrapper to the new Lua function. Something like:

function OldVimFunction
    lua require("r.module").new_lua_function()
endfunction

Of course, this will be superfluous if the old Vim function is no longer called anywhere.

[jalvesaq]

I will stop pushing to nvimrserver at the moment and look at lua mugration ideas.

@she3o, I never used the interface Lua-C, but from what I could quickly see, there is room for optimization. Currently, we have the following:

• Completion data is saved by nvimcom in a custom simple binary format that separates fields and replaces single quotes with bytes that hopefully are never used.
• Nclientserver reads this data, and converts it into a string that Vim can evaluate as a valid dictionary.
• This Vim dictionary is automatically converted to a Lua table because it is used in Lua code (cmp-nvim-r).

Perhaps we may keep the first step as is, but:

• Nclientserver would be a library, and not a standalone application. It would convert the data directly into a Lua table ready to be used by cmp-nvim-r.
• This table would be sent to cmp-nvim-r directly, as a call from C to Lua.

Is this possible? Would it be an improvement?

[she3o]

I am wondering if keeping nvimr server would be necessary if we migrate to lua. Lua seems to have a wonderful luasocket library that handles TCP server/client architecture.

The nvimrserver.c file involves other parts not related to tcp communication. It was the same parts I was having trouble with in my PR.

[jalvesaq]

I guess it would be better if we could use the Lua library for the TCP server. But we will still need the TCP client in nvimcom.

What parts are problematic?

[jalvesaq]

@akthe-at and @hongyuanjia : tmp-R-Nvim is finished and almost completely tested on Unix systems. It would be great if you could try it on Windows and fix the bugs that certainly it has. The README has a section highlighting the differences between Nvim-R and tmp-R-Nvim.

[hongyuanjia]

Super! I will try migrating my NVim-R config to R.nvim and check if everything works.

[akthe-at]

I tried to open it quick this morning before heading out for my daughter's birthday but encountered an error. I look forward to trying to debug it and submit any fix I find. Fingers crossed since I'm not super well versed in lua (only experience is configs for wezterm and neovim) thanks for all the work you all put in on this.

[jalvesaq]

Some links in the README of tmp-R-Nvim were pointing to yet non-existent addresses. I "updated" them back to the present. Only "R-nvim/cmp-r" already exists.

[LiNk-NY]

I'd like to contribute. I am an R developer and can help with any R code, if needed. Thanks for working on this!

[akthe-at]

I am 3 hours deep into testing on Windows and haven't run into any issues yet. Everything seems much more responsive and snappier than nvim-r. Que bom, @jalvesaq ! ...Stay tuned in case I do find some issues beyond the whole neovim 9.5 vs nightly issue.

[jalvesaq]

@hongyuanjia adapted the code to work with Neovim 0.9.5: R-nvim/R.nvim#41

[akthe-at]

Okay! Just switched back to neovim 0.9.5 since that is what I normally run on the daily.

[jalvesaq]

I'd like to contribute. I am an R developer and can help with any R code, if needed. Thanks for working on this!

Thanks! As usual with free software, you can test features, develop new ones, fix bugs, or optimize code. Currently, we are working at https://github.com/jalvesaq/tmp-R-Nvim

[akthe-at]

I ran half of my work day today on neovim nightly and half the day with 0.9.5. I encountered zero bugs or errors besides the moment when I switched back to 0.9.5 but had not yet installed the fix for that. Moments later after lazy.nvim updated I was good for the rest of the day.

I am running in Windows 10, no WSL using a variety of custom packages, odbc connections, tidyverse, and base R throughout the day. It was very pleasant actually.

[jalvesaq]

Time to close this discussion:
https://github.com/R-nvim/R.nvim

[PMassicotte]

Congratulation!

[jalvesaq]

The merit is yours too.

[PMassicotte]

Tx! I was busy this week with job, but will finish the send-function PR this weekend.

[wurli]

🥳 Is it worth adding a note to Nvim-R's README directing people towards R.nvim?

[jalvesaq]

It has in the README: "This is Nvim-R which improves Vim's support to edit R scripts. For Neovim users, this plugin was superseded by R.nvim."

Also in R/start_nrs.vim:

Nvim-R/R/start_nrs.vim

Line 533 in 82b8468

call RWarningMsg('Nvim-R was superseded by https://github.com/R-nvim/R.nvim')

[wurli]

Thank you, not sure how I missed that!