mmm-mode with typescript-ts-mode is very slow

[lesteral]

Hello,

I have used mmm-mode for a while (thank you very much), together with typescript-mode.

However, I have encountered a major slowdown when combining mmm-mode with typescript-ts-mode.
I haven't been able to solve this myself, and would appreciate any advice or guidance which you may have.

I am using emacs-29.1 (https://packages.debian.org/bookworm-backports/emacs when it was at 1:29.1+1-5~bpo12+1) and mmm-mode 0.5.11.

I have simplified to use this ~/.emacs.el:

(add-to-list 'load-path "~/emacs")  ; for customized "treesit.el"

(require 'mmm-auto)
(setq mmm-global-mode 'maybe)
(mmm-add-classes '((html-ts :submode html-mode :front "^ *template: *`" :back "`")))
(mmm-add-mode-ext-class 'typescript-ts-mode "\\.ts\\'" 'html-ts)

(require 'typescript-ts-mode)
(add-to-list 'auto-mode-alist '("\\.ts\\'" . typescript-ts-mode))

and this test.ts Typescript file (Angular syntax: the "template" section is HTML):
test.ts.txt

@Component({
  template: `
    <h4>test</h4>
`,
})
export class TestComponent { }

I have customized treesit.el with the one-line fix in #138 (comment)

(I also tried https://github.com/emacs-mirror/emacs/blob/emacs-29.3/lisp/treesit.el but encountered the same behavior.)

When I use emacs profiler-start ... profiler-stop ... profiler-report, and move (up/down arrow) in and out of the "template" code (h4 test /h4), I see:

         525  54% - ...
         525  54%    Automatic GC
         352  36% - command-execute
         315  32%  - byte-code
         315  32%   - read-extended-command
         315  32%    - read-extended-command-1
         276  28%     - completing-read-default
         254  26%      - redisplay_internal (C function)
         253  26%       - jit-lock-function
         253  26%        - jit-lock-fontify-now
         253  26%         - jit-lock--run-functions
         253  26%          - #<compiled -0x156e73e7fc33e483>
         253  26%           - font-lock-fontify-region
         253  26%            - mmm-fontify-region
         252  26%             - #<compiled 0xf68e4846fb4d198>
         252  26%              - mmm-fontify-region-list
         252  26%               - #<compiled 0x1397da0f1021fd80>
         251  26%                - font-lock-default-fontify-region
         251  26%                 - font-lock-fontify-syntactically-region
         251  26%                  - treesit-font-lock-fontify-region
         249  26%                   - let
         249  26%                    - while
         249  26%                     - let
         247  25%                      - let*
         240  25%                       - let*
         193  20%                        - if
         193  20%                         - progn
         193  20%                          - let
         193  20%                           - while
         193  20%                            - let
         189  19%                             - let*
         139  14%                              - let*
         137  14%                               - unwind-protect
         136  14%                                - progn
         130  13%                                 - let
         130  13%                                  - while
         130  13%                                   - let
         129  13%                                    - let*
         129  13%                                     - if
         122  12%                                      - if
         122  12%                                       - progn
           1   0%                                          message
           7   0%                                      + cond
           1   0%                                + if
           3   0%                              + treesit-query-capture
          46   4%                        + and
           2   0%                       + if
           2   0%                   + if
           1   0%                  mmm-set-local-variables
           1   0%             + mmm-regions-alist
           1   0%         redisplay--pre-redisplay-functions
           3   0%      + timer-event-handler
          36   3%  + funcall-interactively
          42   4% + timer-event-handler
          37   3% - redisplay_internal (C function)
          30   3%  - jit-lock-function
          29   3%   - jit-lock-fontify-now
          29   3%    - jit-lock--run-functions
          29   3%     - #<compiled -0x156e7391761d5083>
          29   3%      - font-lock-fontify-region
          29   3%       - mmm-fontify-region
          26   2%        - #<compiled 0xf68e4846fd0a198>
          24   2%         - mmm-fontify-region-list
          24   2%          - #<compiled 0x1397db5ce76ffd80>
          22   2%           - font-lock-default-fontify-region
          19   1%            - font-lock-fontify-syntactically-region
          19   1%             - treesit-font-lock-fontify-region
          17   1%              - if
          17   1%                 progn
           1   0%             mmm-set-local-variables
           2   0%        + mmm-regions-alist
           1   0%        + #<compiled 0xa0ee66f77b032a6>
           3   0%  + redisplay--pre-redisplay-functions
           2   0%  + eval
           1   0%  + mode-line-default-help-echo

With a larger .ts file (several hundred lines of HTML in the template), emacs becomes almost unresponsive.
A profiler report for that case is here:

        1163  88% - redisplay_internal (C function)
        1159  88%  - jit-lock-function
        1159  88%   - jit-lock-fontify-now
        1158  87%    - jit-lock--run-functions
        1158  87%     - #<compiled -0x156f5e65c219f483>
        1158  87%      - font-lock-fontify-region
        1157  87%       - mmm-fontify-region
        1156  87%        - #<compiled 0xf68e48502e584d8>
        1155  87%         - mmm-fontify-region-list
        1154  87%          - #<compiled -0x86aabab82b5e17d>
        1153  87%           - font-lock-default-fontify-region
        1146  87%            - font-lock-fontify-syntactically-region
        1146  87%             - treesit-font-lock-fontify-region
        1144  86%              - let
        1144  86%               - while
        1143  86%                - let
        1141  86%                 - let*
        1141  86%                  - let*
         942  71%                   - and
         941  71%                    - treesit-buffer-root-node
         941  71%                     - let*
         941  71%                      - if
          26   1%                       - treesit-parser-root-node
          25   1%                        - treesit--font-lock-notifier
          25   1%                         - save-current-buffer
          25   1%                          - let
          25   1%                           - while
          25   1%                            - let
          20   1%                             + let*
           5   0%                             + if
           1   0%                        - treesit--syntax-propertize-notifier
           1   0%                         + save-current-buffer
         199  15%                   + if
           1   0%                font-lock-unfontify-region
           1   0%              + if
           1   0%              font-lock-unfontify-region
           1   0%             mmm-set-local-variables
           1   0%          #<compiled 0x2c4d5446d479b48>
           2   0%    redisplay--pre-redisplay-functions
           1   0%    kill-this-buffer-enabled-p
           1   0%  + mode-line-default-help-echo
          83   6% + command-execute
          63   4% + ...
           6   0% + timer-event-handler
           1   0% + mmm-update-submode-region

I also enabled "LOUDLY" in treesit.el by changing the below variable from default (nil) to t:

(defvar treesit--font-lock-verbose t
  "If non-nil, print debug messages when fontifying.")

When mmm-mode is not enabled (with the above test.ts file), the Messages buffer contains just:

...
For information about GNU Emacs and the GNU system, type C-h C-a.
Fontifying region: 1-83
Fontifying text from 52 to 58, Face: font-lock-keyword-face, Node: export
Fontifying text from 59 to 64, Face: font-lock-keyword-face, Node: class
Fontifying text from 26 to 47, Face: js--fontify-template-string, Node: template_string
Fontifying text from 65 to 78, Face: font-lock-type-face, Node: type_identifier
Fontifying text from 16 to 24, Face: font-lock-property-use-face, Node: property_identifier

However, with mmm-mode enabled for this file, the Messages buffer continuously streams "LOUDLY" messages (tail shown here):

...
Captured node #<treesit-node property_identifier in 16-24>(16-24) but it is outside of fontifing region
Fontifying region: 33-46
Notifier received range: 27-46
Fontifying text from 33 to 35, Face: font-lock-type-face, Node: type_identifier
Fontifying region: 27-46
Fontifying text from 33 to 35, Face: font-lock-type-face, Node: type_identifier
Fontifying region: 16-27
Notifier received range: 1-82
Fontifying text from 26 to 27, Face: js--fontify-template-string, Node: template_string
Fontifying text from 16 to 24, Face: font-lock-property-use-face, Node: property_identifier
Fontifying region: 46-82
Fontifying text from 52 to 58, Face: font-lock-keyword-face, Node: export
Fontifying text from 59 to 64, Face: font-lock-keyword-face, Node: class
Fontifying text from 46 to 47, Face: js--fontify-template-string, Node: template_string
Fontifying text from 65 to 78, Face: font-lock-type-face, Node: type_identifier
Captured node #<treesit-node property_identifier in 16-24>(16-24) but it is outside of fontifing region
Fontifying region: 27-46
Notifier received range: 27-46
Fontifying text from 33 to 35, Face: font-lock-type-face, Node: type_identifier
Fontifying region: 24-27
Notifier received range: 1-82
Fontifying text from 26 to 27, Face: js--fontify-template-string, Node: template_string
Captured node #<treesit-node property_identifier in 16-24>(16-24) but it is outside of fontifing region
Fontifying region: 46-82
Fontifying text from 52 to 58, Face: font-lock-keyword-face, Node: export
Fontifying text from 59 to 64, Face: font-lock-keyword-face, Node: class
Fontifying text from 46 to 47, Face: js--fontify-template-string, Node: template_string
Fontifying text from 65 to 78, Face: font-lock-type-face, Node: type_identifier
Captured node #<treesit-node property_identifier in 16-24>(16-24) but it is outside of fontifing region
Fontifying region: 27-46
Notifier received range: 27-46
Fontifying text from 33 to 35, Face: font-lock-type-face, Node: type_identifier
Fontifying region: 26-27
Notifier received range: 1-82
Fontifying text from 26 to 27, Face: js--fontify-template-string, Node: template_string
Captured node #<treesit-node property_identifier in 16-24>(16-24) but it is outside of fontifing region
Fontifying region: 46-82
Fontifying text from 52 to 58, Face: font-lock-keyword-face, Node: export
Fontifying text from 59 to 64, Face: font-lock-keyword-face, Node: class
Fontifying text from 46 to 47, Face: js--fontify-template-string, Node: template_string
Fontifying text from 65 to 78, Face: font-lock-type-face, Node: type_identifier
Captured node #<treesit-node property_identifier in 16-24>(16-24) but it is outside of fontifing region
Fontifying region: 27-46
Notifier received range: 27-46
Fontifying text from 33 to 35, Face: font-lock-type-face, Node: type_identifier
Fontifying region: 33-46
Fontifying text from 33 to 35, Face: font-lock-type-face, Node: type_identifier

At a high level, it seems to me that the fontifying running continuously (with mmm-mode enabled) is what creates such a heavy load.

If you have any suggestions on how to narrow this down, or improve the behavior, please advise.

Thanks very much,
Lester

[dgutov]

Hi!

tree-sitter does things a bit differently, and mmm-mode also has a peculiarity in how it reparses buffers (a certain implementation shortcut), so it's no surprise that large buffers can have performance problems with this combination.

I'll look into it when I have more time, but before that, you can try setting up the native tree-sitter support for mixed languages (the "ranges" thing). Here's some documentation for it:

https://www.gnu.org/software/emacs/manual/html_node/elisp/Multiple-Languages.html

Note that this feature is also very new. For best results try Emacs 29.3 or newer.

[lesteral]

Dmitry - thanks for the quick feedback and for the pointer to the tree-sitter "ranges" feature (of which I was unaware; I'll see how far I can get with that.) -Lester

[lesteral]

Dmitry - the tree-sitter "ranges" feature is an effective workaround for my purposes. mmm-mode is nice w.r.t. truly switching modes, but the tree-sitter font-locking for multiple languages gets me most of the way there. Thanks again, Lester

[dgutov]

Glad it's working out for you!

Perhaps you'll want to paste your config for typescript ranges here, for anybody having this problem in the meantime while this is unfixed.

[lesteral]

Hi Dmitry,

Sure/thanks - see below.

A few notes:

1. I installed https://github.com/mickeynp/html-ts-mode/blob/master/html-ts-mode.el and used its html-ts-font-lock-rules below - idea from https://www.masteringemacs.org/article/lets-write-a-treesitter-major-mode#:~:text=you%20can%20safely%20append%20to%20treesit%2Dfont%2Dlock%2Dsettings%20at%20any%20point
2. I found Mickey's "combobulate" package to be helpful in learning to build the treesitter query.
3. I'm using 'v0.20.1'-tagged version of https://github.com/tree-sitter/tree-sitter-html (since there's been some churn in this area)
4. I'm using 'emacs-29.3'-tagged version of treesit.el (i.e., https://github.com/emacs-mirror/emacs/blob/emacs-29.3/lisp/treesit.el) with emacs-29.1 (this combo. seems to work, albeit hacky; I don't know that it's necessary, however)

(add-to-list 'load-path "~/emacs")  ; html-ts-mode.el installed here

(require 'typescript-ts-mode)
(require 'html-ts-mode)

(add-to-list 'auto-mode-alist '("\\.ts\\'" . typescript-ts-mode))

(add-hook 'typescript-ts-mode-hook
  (lambda ()
    (when (treesit-available-p)
      (when (treesit-ready-p 'html)
        (setq treesit-range-settings
          (treesit-range-rules
            (lambda (beg end)
              (treesit-parser-set-included-ranges (treesit-parser-create 'html)
                (seq-map
                  (lambda (elt) (let ((node (cdr elt))) (cons (1+ (treesit-node-start node)) (1- (treesit-node-end node)))))  ; using "1+" & "1-" to skip leading/trailing backquotes
                  (seq-filter (lambda (elt) (eq (car elt) 'templ))
                    (treesit-query-capture 'typescript '(((pair key: (property_identifier) @propid value: (template_string) @templ) (:equal "template" @propid))) beg end) ))))))

        (setq treesit-font-lock-settings (append (apply #'treesit-font-lock-rules html-ts-font-lock-rules) treesit-font-lock-settings))
        ))))

[dgutov]

Looking good!

I'm using 'emacs-29.3'-tagged version of treesit.el (i.e., https://github.com/emacs-mirror/emacs/blob/emacs-29.3/lisp/treesit.el) with emacs-29.1 (this combo. seems to work, albeit hacky; I don't know that it's necessary, however)

I would keep using it (if you can't switch to 29.3 entirely). There have been fixes related to ranges past 29.1, though I don't remember the exact details.

[lesteral]

Hi Dmitry,

Thanks, I've now moved to emacs 29.3, facilitated by its recent arrival in Debian 12 backports.

For completeness & in case it's helpful to someone, regarding the approach I listed in the previous comment in this thread,
I'd like to add that I found I also needed to include the following configuration, in typescript-ts-mode-hook,
in order for indenting to work properly in Typescript code:

(setq-local treesit-language-at-point-function
  (lambda (pos)
    (if (string= "template_string" (treesit-node-type (treesit-node-parent (treesit-node-at pos 'typescript))))
        'html
      'typescript)
    ))

Otherwise, this behavior of treesit-language-at:

... It returns the return value of treesit-language-at-point-function if it’s non-nil,
otherwise it returns the language of the first parser in treesit-parser-list, or nil if there is no parser.

which otherwise resulted, for me, in treesit-language-at typically returning 'html,
when it should return 'typescript; this operation effectively breaks the indenting.
(The treesit-parser-list is: 'html, 'typescript, and so 'html is the "first" parser.)

I also found it useful to add 'html indenting rules to treesit-simple-indent-rules,
so that indenting also works in the HTML template strings.
I found that Mickey's rules work well: https://github.com/mickeynp/html-ts-mode/blob/master/html-ts-mode.el#:~:text=%28setq%2Dlocal-,treesit%2Dsimple%2Dindent%2Drules

Regards,
Lester

[dgutov]

I'd like to add that I found I also needed to include the following configuration, in typescript-ts-mode-hook,
in order for indenting to work properly in Typescript code:

Yep, that makes sense.

The manual page I originally linked to actually mentions this (the beginning of the page, second paragraph near the end), but the code examples don't, so it's easy enough to miss. I'll file a bug.

Maybe you'll want to publish a public gist for "Angular templating setup with typescript-ts-mode"? Does that sound like an appropriate title?

[lesteral]

Hi Dmitry,

Thanks, I see that reference now, and indeed it would be helpful to have that aspect considered in the examples.

A gist is an excellent way to provide the foregoing info--thanks for the suggestion.
I'll do that (your title sounds appropriate) & post the link here, for closure.

Regards,
Lester

[dgutov]

Now added: emacs-mirror/emacs@e947e63b066

[lesteral]

Thanks - and here is the gist (I thought I had posted it earlier, sorry): Angular templating setup with typescript-ts-mode