Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better syntax highlighting: switching from highlight.js to shiki #2467

Open
julio4 opened this issue Nov 4, 2024 · 0 comments
Open

Better syntax highlighting: switching from highlight.js to shiki #2467

julio4 opened this issue Nov 4, 2024 · 0 comments
Labels
C-enhancement Category: Enhancement or feature request

Comments

@julio4
Copy link

julio4 commented Nov 4, 2024

Problem

The current highlighting library highlight.js is lightweight and can run on both server and client, but provides a very basic syntax parsing that could be greatly improved.

Supersede #2313 #2292 #2268 #2238 #2237 #2360 #2445 #1652 ...

Proposed Solution

Consider switching to shiki, a high quality syntax highlighter that uses TextMate grammars and themes (same as VS Code), yet stays performant. It is also actively maintained.

There have been significant efforts and improvements that led to shiki-v1, and I believe that it's worthwile to make this migration and make shiki the default highlighter for mdbook. Today, mdbook is used to document a lot of code snippets and this change would definitely improve readability.

Example

Let's take Listing 20-24 from the rust book:

impl Worker {
    fn new(id: usize, receiver: Arc<Mutex<mpsc::Receiver<Job>>>) -> Worker {
        let thread = thread::spawn(move || loop {
            let message = receiver.lock().unwrap().recv();

            match message {
                Ok(job) => {
                    println!("Worker {id} got a job; executing.");

                    job();
                }
                Err(_) => {
                    println!("Worker {id} disconnected; shutting down.");
                    break;
                }
            }
        });

        Worker {
            id,
            thread: Some(thread),
        }
    }
}
mdbook w/ highlight.js (Coal theme) Shiki (github-dark-dimmed theme)
Open in browser Open in browser
Image Image

Vs-code reference (theme Github Dark Dimmed):
Image

Implementation example

@cestef overriding of theme/book.js for shiki compatibility as explained here

Notes

Shiki v1: https://nuxt.com/blog/shiki-v1
Another alternative: https://github.com/wooorm/starry-night

Further ideas

Twoslash allows to extend Typescript code snippets from markdown file with extended Typescript compiler's information and vs code syntax engine. It is maintained and integrated perfectly with shiki.

With the strong rust-analyzer and the recent abstraction of the twoslash API in twoslash-protocol, it would make sense to build a twoslash-rs for mdbook. Some preliminary explorations have been done: by ayazhafiz/twoslash-rust and @jxom

This first implementation of Shiki in mdbook would pave the way for this later on.

@julio4 julio4 added the C-enhancement Category: Enhancement or feature request label Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Category: Enhancement or feature request
Projects
None yet
Development

No branches or pull requests

1 participant