Write an (online) book about writing interpreters in Rust

[yorickpeterse]

By providing a book of sorts (e.g. using https://www.gitbook.com/) we would solve two problems:

1. With the book focusing on Rust we'd hopefully make Rust more attractive for language designers
2. By clearly explaining how to implement certain patterns it will hopefully become easier for everybody to write a language

The book wouldn't focus on writing an interpreter from start to finish (that's a lot of work), instead it would focus more on specific patterns (e.g. how do I write an interpreter loop, how do I come up with a bytecode format, etc, etc). For the sake of keeping the scope limited we should probably start with an interpreter, but maybe over time we can also extend it to writing compilers.

Some topics to cover would be (in no particular order):

1. How to write an interpreter loop
2. Writing allocators for a custom object model/heap
3. Basic garbage collection (naïve mark & sweep)
4. Advanced garbage collection (Immix)
5. Parsing/using a bytecode format
6. Handling errors triggered by the programming language (e.g. an alternative to a straight up panic!)
7. Concurrency (e.g. regular threads, green threads)

I'm not sure if we should cover parsing. Unfortunately many of existing programming language resources fall for the trap of only covering parsing, which I think is one of the most boring parts of a programming language. It's also very easy to get stuck bike-shedding about the syntax and how to parse it, instead of just focusing on writing an interpreter/allocator/etc.

Any thoughts?

[pliniker]

This is a great idea, agree 100% on all points!

I think this might be what #1 is missing in terms of a specific purpose and focus. As I think you mentioned in a conversation elsewhere about Inko, language-specific allocator, GC and bytecode optimizations happen, which will lead to some different Rust code patterns and tradeoffs across different languages (imperative vs purely functional as the obvious example.) However, I think the points you've outlined above are a good place to start and other language implementations could be covered later.

Parsing is a well covered and less interesting topic, but it might be worthwhile to integrate all the basic-level topics - mark & sweep, bytecode interpreter loop, all on a single thread - with a simple s-expr to bytecode compiler, for completeness. That avoids syntax bikeshedding and keeps parsing as simple as possible.

I would add one more topic: "Exposing access to managed-objects outside of the VM environment" for covering different ways and tradeoffs of interacting with the VM and heap in safe Rust.

For arranging a book, there's also https://github.com/rust-lang-nursery/mdBook fwiw, but I don't have experience with either mdBook or GitBook.

[pliniker]

I've created https://github.com/rust-hosted-langs/book for editing a book in. I took a look at mdBook, it's straightforward to use and importantly, has excellent Rust code-snippet support.

We can use the book label in this repo to group discussions on content. Perhaps creating one issue per chapter to cover would make sense?

[yorickpeterse]

@pliniker

We can use the book label in this repo to group discussions on content. Perhaps creating one issue per chapter to cover would make sense?

That sounds like a good plan. One thing to keep in mind is that at least early on we probably shouldn't focus too much on the organisation/structure. Doing so can lead to a lot of arguing about what goes where, without actually having any content to organise in the first place.

[pliniker]

Just to keep this issue consistent, discussion of book content is now on the issue tracker for the book repo:

• https://github.com/rust-hosted-langs/book/issues