State of Tau and its new C VM.

[NicoNex]

After five months, the rewrite of the Tau virtual machine in C is almost done.
As I've mentioned in the old discussion, the experimental CTau repository is discontinued as there's no need to rewrite each part of the language in C thanks to Go's awesome CGO feature.
So the project will mostly stay in our beloved Go but with a portable C heart.

The next major release of Tau will include the C VM as part of the main repository and will disable the evaluator, which will be only used for some compiler optimizations.
All the C code in Tau has been written with speed and portability in mind, for more details and benchmark results refer to the bottom of this comment.
I'm currently considering disabling or removing the old Go VM alltogether since it has been greatly outperformed by the C implementation (roughly 5x faster).

The development of the C VM can be followed in the fast-vm branch.
At the moment it's still missing a few builtins and an implementation of the pipe object.
The missing builtins are:

• push (removed)
• range (removed)
• plugin (might change)
• pipe
• send
• recv
• close
• open (removed)
• bytes

The major changes to the VM are:

• A rewritten bytecode evaluation loop, which now makes use of an explicit static jump table and an explicit indirect jump instruction (instead of a switch case). This allows to get more benefits from the CPU cache that will have higher chances to predict the following instructions and thus waste way less cycles by not having to flush the pipeline after almost every instruction.
• A rewritten internal object system that creates the least heap allocations possible and works mostly on the stack. In this way, objects not only have a very lightweight memory footprint but, due to the usage of a union, they avoid the expensive interface castings that would happen in the Go implementation.
• A new concurrent and fast mark-and-sweep garbage collector. Since the object system and the VM are now in C, we need to handle memory explicitly as we cannot rely anymore on the internal gc of Go.
• A new, way faster bytecode codec (encoder/decoder) and layout, that produces precompiled Tau files up to 3x smaller than the previously used GOB library.

Up-to-date benchmarks on Linux 6.2.9 running on an Intel Core i7 8700K (x86_64):
CTau: 10,899s
CPython 3.11.2: 13,231s
CPython 3.10.10: 20,397s
GoTau: 53,563s

Benchmarking Tau code:

fib = fn(n) {
    if n < 2 {
        return n
    }
    fib(n-1) + fib(n-2)
}

println(fib(40))

Benchmarking Python code:

def fib(n):
    if n < 2:
        return n
    return fib(n-1) + fib(n-2)

print(fib(40))

[NicoNex]

If you have any question or suggestion that you would like to share don't hesitate to write here. 😃

[prologic]

Nice one 👌