lag while scrolling using default configuration

[tklebanoff]

#Description of issue

I load the plugin via Packer

use { 'Raku/vim-raku' }

I run :PackerUpdate.

The next time I launch vim, it looks like everything is working except that I experience massive lag while scrolling up and down in the buffer. Sometimes I also get the lag when entering new text.

The lag feels like up to a second for each keystroke. It is prohibitive to any sort of editing raku files.

File sizes are only a few hundred lines.

Here is the output of syntime report after scrolling up and down for a bit on a 160 line raku file:
https://termbin.com/stjp

Output of vim --version

NVIM v0.7.0-dev+1135-gfdea15723
Build type: Release
LuaJIT 2.1.0-beta3
Compiled by [email protected]

Features: +acl +iconv +tui
See ":help feature-compile"

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/usr/local/Cellar/neovim/HEAD-fdea157/share/nvim"

#profile output
https://termbin.com/wd9x

[tklebanoff]

I compare the grammar benchmark to one for another language (in this case, rust -- much faster) with a similar match count
https://termbin.com/7kjl

[tklebanoff]

one thing i notice (found in the profile output), is that we spend quite a bit of time in highlight_match_pair

FUNCTIONS SORTED ON TOTAL TIME
count  total (s)   self (s)  function
 1304   9.013860   8.962009  <SNR>60_Highlight_Matching_Pair()
   16   2.091379   0.169232  <SNR>1_LoadFTPlugin()
   16   0.223840   0.070922  <SNR>8_SynSet()
 2653   0.062753             <SNR>139_compareReverseFtime()
 1318   0.052102             <SNR>60_Remove_Matches()
   16   0.031767   0.027725  <SNR>2_LoadIndent()
    7   0.012497   0.000502  FugitiveDetect()
    7   0.008363   0.001653  fugitive#Init()
    7   0.003633   0.002679  FugitiveExtractGitDir()
   14   0.002450             <SNR>138_map()
    7   0.002341             <SNR>138_define_commands()
    7   0.001133             <SNR>11_isAnsible()
   28   0.000845             FugitiveIsGitDir()
   21   0.000635             nerdtree#checkForBrowse()
    7   0.000611   0.000314  <SNR>138_SetupTemp()
    1   0.000509   0.000133  <SNR>138_AutoReloadStatus()
    7   0.000371   0.000279  <SNR>138_DirCommitFile()
   14   0.000336             <SNR>37_Swift()
    1   0.000331             fugitive#ReloadStatus()
    7   0.000297   0.000229  <SNR>138_cpath()

FUNCTIONS SORTED ON SELF TIME
count  total (s)   self (s)  function
 1304   9.013860   8.962009  <SNR>60_Highlight_Matching_Pair()
   16   2.091379   0.169232  <SNR>1_LoadFTPlugin()
   16   0.223840   0.070922  <SNR>8_SynSet()
 2653              0.062753  <SNR>139_compareReverseFtime()
 1318              0.052102  <SNR>60_Remove_Matches()
   16   0.031767   0.027725  <SNR>2_LoadIndent()
    7   0.003633   0.002679  FugitiveExtractGitDir()
   14              0.002450  <SNR>138_map()
    7              0.002341  <SNR>138_define_commands()
    7   0.008363   0.001653  fugitive#Init()
    7              0.001133  <SNR>11_isAnsible()
   28              0.000845  FugitiveIsGitDir()
   21              0.000635  nerdtree#checkForBrowse()
    7   0.012497   0.000502  FugitiveDetect()
   14              0.000336  <SNR>37_Swift()
    1              0.000331  fugitive#ReloadStatus()
    7   0.000611   0.000314  <SNR>138_SetupTemp()
    7   0.000371   0.000279  <SNR>138_DirCommitFile()
   14              0.000257  <SNR>138_can_diffoff()
    7   0.000297   0.000229  <SNR>138_cpath()

[tklebanoff]

here are the first few lines of the raku :syntime report

 TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN

  4.643822   1283   0       0.035831    0.003620  rakuStringAuto     
  \.\@1<!\%([A-Za-z_\xC0-\xFF]\%([A-Za-z_\xC0-\xFF0-9]\|[-'][A-Za-z_\xC0-\xFF]\@=\)*\)\ze\%(p5\)\@2<![RSXZ]\@1<!=>

  0.967684   1298   30      0.004870    0.000746  rakuNumber         
  [A-Za-z_\xC0-\xFF0-9]\@1<!\%(\%(\_^\|\s\|[^*\a]\)\@1<=[-+]\)\?\%(\%(\d\|__\@!\)*[._]\@1<!\.\)\?_\@!\%(\d\|_\)\+_\@1<!\%([eE]-\?_\@!\%(\d\|_\)\+\)\?i\?

  0.959792   1283   0       0.005042    0.000748  rakuTypeConstraint 
  \%([.^]\|^\s*\)\@<![A-Za-z_\xC0-\xFF0-9]\@1<!\%([A-Za-z_\xC0-\xFF][-']\)\@2<!is\%([A-Za-z_\xC0-\xFF0-9]\|[-'][A-Za-z_\xC0-\xFF]\)\@!

  0.866801   1579   0       0.003788    0.000549  rakuMatchBare      
  /\@1<!\%(\%(\_^\|[!\[,=~|&/:({]\|\^\?fff\?\^\?\|=>\|\<\%(if\|unless\|while\|when\|where\|so\)\)\s*\)\@<=/[/=]\@!

  0.752256   3180   1927    0.002983    0.000237  rakuKeywordStart   
  \%(\%([A-Za-z_\xC0-\xFF]\%([A-Za-z_\xC0-\xFF0-9]\|[-'][A-Za-z_\xC0-\xFF]\@=\)*\)\%([A-Za-z_\xC0-\xFF0-9]\|[-'][A-Za-z_\xC0-\xFF]\)\@!\)\@=[A-Za-z_\xC0-\xF

  0.729245   1283   0       0.004003    0.000568  rakuNumber         
  [A-Za-z_\xC0-\xFF0-9]\@1<!\%(\%(\_^\|\s\|[^*\a]\)\@1<=[-+]\)\?0[obxd]\@=

  0.695617   1283   0       0.004114    0.000542  rakuType           
  \%(::\)\@2<!\%(SeekType\%(::SeekFromBeginning\|::SeekFromCurrent\|::SeekFromEnd\)\|Order\%(::Same\|::More\|::Less\)\?\|Bool\%(::True\|::False\)\?\)\%([A-Z

  0.682181   1368   28      0.003353    0.000499  rakuIdentifier     
  \%(::\)\@2<=\%([A-Za-z_\xC0-\xFF]\%([A-Za-z_\xC0-\xFF0-9]\|[-'][A-Za-z_\xC0-\xFF]\@=\)*\)*

  0.681406   1475   192     0.004800    0.000462  rakuTypeConstraint 
  \%([.^]\|^\s*\)\@<!\a\@=\%(does\|as\|but\|trusts\|of\|returns\|handles\|where\|augment\|supersede\)\>

  0.454782   1283   0       0.003427    0.000354  rakuNumber         
  [A-Za-z_\xC0-\xFF0-9]\@1<!\%(\%(\%(\_^\|\s\|[^*\a]\)\@1<=[-+]\)\?Inf\|NaN\)

  0.432775   1396   56      0.002872    0.000310  rakuIdentifier     
  \%([A-Za-z_\xC0-\xFF]\%([A-Za-z_\xC0-\xFF0-9]\|[-'][A-Za-z_\xC0-\xFF]\@=\)*\)\%(::\)\@=

  0.363575   601    0       0.004196    0.000605  rakuInclude        
  [.^]\@1<!\%(use\|require\|import\|unit\)(\@!\%([A-Za-z_\xC0-\xFF0-9]\|[-'][A-Za-z_\xC0-\xFF]\)\@!

  0.349422   1283   0       0.002270    0.000272  rakuStringAuto     
  \.\@1<!\%([A-Za-z_\xC0-\xFF]\%([A-Za-z_\xC0-\xFF0-9]\|[-'][A-Za-z_\xC0-\xFF]\@=\)*\)p5\ze=>

[tklebanoff]

and the rust:

  TOTAL      COUNT  MATCH   SLOWEST     AVERAGE   NAME               PATTERN
  0.176871   1182   0       0.000982    0.000150  rustCommentLinesDocRustCode 
  ^\z(\s*//[!/]\s*```\)[^A-Za-z0-9_-]*\%(\%(should_panic\|no_run\|ignore\|allow_fail\|rust\|test_harness\|compile_fail\|E\d\{4}\|edition201[58]\)\%( [^A-Za-z

  0.107862   1162   0       0.001534    0.000093  rustFloat          
  \<[0-9][0-9_]*\%(\.[0-9][0-9_]*\)\=\%([eE][+-]\=[0-9_]\+\)\=\(f32\|f64\)

  0.058487   2384   1248    0.001287    0.000025  rustOperator       
  \%(+\|-\|/\|*\|=\|\^\|&\||\|!\|>\|<\|%\)=\?

  0.055823   1854   692     0.000754    0.000030  rustDecNumber      
  \<[0-9][0-9_]*\%([iu]\%(size\|8\|16\|32\|64\|128\)\)\=

  0.049141   1162   0       0.000679    0.000042  rustFloat          
  \<[0-9][0-9_]*\%(\.[0-9][0-9_]*\)\%([eE][+-]\=[0-9_]\+\)\=\(f32\|f64\)\=

  0.031895   1162   0       0.000821    0.000027  rustLabel          
  \%(\<\%(break\|continue\)\s*\)\@<=\'\%([^[:cntrl:][:space:][:punct:][:digit:]]\|_\)\%([^[:cntrl:][:punct:][:space:]]\|_\)*

  0.023126   1182   0       0.000690    0.000020  rustCommentBlockDocRustCode 
  ^\z(\%(\s*\*\)\?\s*```\)[^A-Za-z0-9_-]*\%(\%(should_panic\|no_run\|ignore\|allow_fail\|rust\|test_harness\|compile_fail\|E\d\{4}\|edition201[58]\) \%([^A-Z

  0.022369   1162   0       0.000640    0.000019  rustFloat          
  \<[0-9][0-9_]*\%(\.[0-9][0-9_]*\)\=\%([eE][+-]\=[0-9_]\+\)\(f32\|f64\)\=

  0.018462   1260   98      0.000751    0.000015  rustLifetimeCandidate 
  &'\%(\([^'\\]\|\\\(['nrt0\\\"]\|x\x\{2}\|u{\%(\x_*\)\{1,6}}\)\)'\)\@!

  0.017779   2131   955     0.000969    0.000008  rustFuncCall       
  \w\(\w\)*(

  0.015714   1182   0       0.000961    0.000013  rustString         
  b\?r\z(#*\)"

  0.015679   1295   113     0.000555    0.000012  rustMacro          
  \w\(\w\)*!

  0.014226   1182   0       0.000250    0.000012  rustKeyword        
  \%(\<impl\>.\+\)\@<=\<for\>

  0.014074   1162   0       0.000500    0.000012  rustFloat          
  \<[0-9][0-9_]*\.\%([^[:cntrl:][:space:][:punct:][:digit:]]\|_\|\.\)\@!

  0.013548   1182   0       0.000639    0.000011  rustDefault        
  \<default\ze\_s\+\(impl\|fn\|type\|const\)\>

  0.013538   1303   121     0.000228    0.000010  rustModPath        
  \w\(\w\)*::[^<]

  0.013410   1182   0       0.000288    0.000011  rustFuncCall       
  \w\(\w\)*::<

  0.012465   1182   0       0.000328    0.000011  rustCommentLinesDocNonRustCode 
  ^\z(\s*//[!/]\s*```\).\+$

[tklebanoff]

i suppose the questions I have are the following:

• is this expected behavior?
• if not, what is the expected behavior and how do I access it?
• if yes, how do we improve performance? (ie, which regex construction need to be optimized? how do we optimize them?)

[tklebanoff]

i found a workaround:

  au BufRead,BufNewFile */perl6/site*   set filetype=python
  au BufRead,BufNewFile *.pm6,*.p6,*.t6,*.pod6,*.raku,*.rakumod,*.rakudoc,*.rakutest set filetype=python

[AN3223]

I found MatchParen to be a big contributor to this lag, as a workaround I put autocmd FileType raku NoMatchParen in my vimrc. let g:matchparen_timeout = 5 works OK too.