Custom tree-sitter grammars.

[siph]

Is it possible to add custom grammars for tree-sitter? I'm trying to add
syntax highlighting for nushell.

I see pkgs.tree-sitter has buildGrammar. Can I use that to build a derivation
and somehow append that to grammarPackages and enable through
languageRegister?

[GaetanLepage]

Hello !
Indeed that should be possible. Nixvim allows you to get TS grammars in two ways:

1. By compiling them at runtime thanks to the nvim-treesitter plugin. This is what people do outside of the nix world.
   To adopt this method, set nixGrammars to false and optionally list the grammars that you want in ensureInstalled.
2. Alternatively, as the grammars are made available as derivations in nixpkgs under pkgs.vimPlugins.nvim-treesitter.passthru.allGrammars, you can directly fetch them pre-built.
   This method is the default. However, to avoid downloading all the available grammars, you can list the packages of the grammars you want in grammarPackages.
   Example:

grammarPackages = with config.programs.nixvim.plugins.treesitter.package.builtGrammars; [
  bash
  c
  html
  javascript
  latex
  lua
  nix
  norg
  python
  rust
  vimdoc
];

This being said, I think that the approach you suggests makes sense.
I would be glad to hear if you managed to make it work :)

[siph]

I've had partial success with this. I can see that the parser has compiled and
is added, but the installation isn't complete.

Running :checkhealth shows the compiled parser under vim.treesitter: require("vim.treesitter.health").check().

- OK Parser: nu         ABI: 14, path: /nix/store/r4gpzybxxszi5dxbr5knams5lwfhdkps-vim-pack-dir/pack/myNeovimPackages/start/vimplugin-treesitter-grammar-nu/parser/nu.so

But nvim-treesitter: require("nvim-treesitter.health").check() doesn't list
nu as a module.

I do get string highlights when opening a nu file. I'm not sure why because
running :InspectTree throws an error for no parser. I also don't see the
queries being linked anywhere in the nix store, unlike the parser.

I checked the derivation and it seems like its properly packaged by
buildGrammar.

#/nix/store/rhcmr3plzgsr598l6w843vfc8jlgqff7-nu-grammar-0.0.0+786689b/
 .
├──  parser
└──  queries
    ├──  highlights.scm
    └──  injections.scm

Config:

# treesitter.nix
{pkgs, ...}: {
  plugins.treesitter = {
      enable = true;
      languageRegister.nu = "nu";
      grammarPackages =
        pkgs.vimPlugins.nvim-treesitter.passthru.allGrammars
        ++ [
          (pkgs.callPackage ./nushell.nix {inherit (pkgs.tree-sitter) buildGrammar;})
        ];
    };
  };
}

# nushell.nix
{
  fetchFromGitHub,
  buildGrammar,
}:
buildGrammar {
  language = "nu";
  version = "0.0.0+786689b";
  src = fetchFromGitHub {
    owner = "nushell";
    repo = "tree-sitter-nu";
    rev = "786689b0562b9799ce53e824cb45a1a2a04dc673";
    hash = "sha256-ENyK0l2KhhNfyuXCz0faLwVHMJjJxlRCqitJkJ8fD+w=";
  };
}

[GaetanLepage]

My feeling is that everything is fine on the native treesitter side (vim.treesitter) but somehow the nvim-treesitter plugin is not aware of your parser and thus you don't get the expected features (highlighting...).

[GaetanLepage]

Indeed, by adding

extraConfigLua = ''
  local parser_config = require("nvim-treesitter.parsers").get_parser_configs()
  parser_config.nu = {
    filetype = "nu",
  }
'';

Then the parser is found by nvim-treesitter.

[GaetanLepage]

Here is the config that ended up working for me:

# Automatically set the filetype of "*.nu" files to "nu".
filetype.extension.nu = "nu";

# Tell nvim-treesitter to use the "nu" parser for filetype "nu".
extraConfigLua = ''
  local parser_config = require("nvim-treesitter.parsers").get_parser_configs()

  parser_config.nu = {
    filetype = "nu",
  }
'';

plugins.treesitter = {
  enable = true;

  languageRegister.nu = "nu";

  grammarPackages = [
    (
      pkgs.tree-sitter.buildGrammar {
        language = "nu";
        version = "0.0.0+786689b";
        src = pkgs.fetchFromGitHub {
          owner = "nushell";
          repo = "tree-sitter-nu";
          rev = "786689b0562b9799ce53e824cb45a1a2a04dc673";
          hash = "sha256-ENyK0l2KhhNfyuXCz0faLwVHMJjJxlRCqitJkJ8fD+w=";
        };
      }
    )
  ];
};

I say that it "works" because :Inspect shows me the tree. However, the parser does not seem to expose any capabilities (highlighting, ...)

Things might be worth to upstream:

• The automatic filetype association of *.nu files to the "nu" filetype could be added to vim/neovim.
• The nu parser could be packaged in nixpkgs.

[siph]

Ok, with the parser working I did some digging and I can see that extraFiles can be used to add files to runtimepath. I can grab those .scm files from the derivation and add them using extraFiles, which will enable those features (highlights, folds, etc...).

Here is the fully working solution I have:

# treesitter.nix
{pkgs, ...}: let
  tree-sitter-nu = pkgs.callPackage ./grammars/nushell.nix {inherit (pkgs.tree-sitter) buildGrammar;};
in {
  plugins.treesitter = {
    enable = true;

    languageRegister.nu = "nu";

    grammarPackages =
      pkgs.vimPlugins.nvim-treesitter.passthru.allGrammars
      ++ [tree-sitter-nu];
  };

  extraFiles = {
    "queries/nu/highlights.scm" = builtins.readFile "${tree-sitter-nu}/queries/highlights.scm";

    "queries/nu/injections.scm" = builtins.readFile "${tree-sitter-nu}/queries/injections.scm";
  };

  filetype.extension.nu = "nu";

  extraConfigLua = ''
    local parser_config = require("nvim-treesitter.parsers").get_parser_configs()

    parser_config.nu = {
      filetype = "nu",
    }
  '';
}

# nushell.nix
{
  fetchFromGitHub,
  buildGrammar,
}:
buildGrammar {
  language = "nu";
  version = "0.0.0+786689b";
  src = fetchFromGitHub {
    owner = "nushell";
    repo = "tree-sitter-nu";
    rev = "786689b0562b9799ce53e824cb45a1a2a04dc673";
    hash = "sha256-ENyK0l2KhhNfyuXCz0faLwVHMJjJxlRCqitJkJ8fD+w=";
  };
}

Thanks a lot for your help @GaetanLepage. The all-white text has been extremely fatiguing!

[GaetanLepage]

Well done !
All of this stuff is good knowledge to keep in mind about treesitter custom parsers !

[GaetanLepage]

Depending on the current state of the treesitter parser for nushell, you could think about upstreaming it to nixpkgs :)

[traxys]

You need to upstream it to nvim-treesitter, not nixpkgs. There is an automatic sync of parsers when updating nvim-treesitter

[siph]

A tree-sitter-nu package actually already exists in nixpkgs: NixOS/nixpkgs#226047. But it is old and I'm not sure of the etiquette of remove/replacing it. Also, nushell and its ecosystem is pretty unstable with very frequent breaking changes. Even this newer parser is clearly missing some edge cases with its highlights when parsing some more complex files. Because of all of this I am hesitant about upstreaming without waiting to see how things stabilize/develop.