future rust implementation for zos

[muhamadazmy]

This is a work in progress description

Goal

ZOS runs many separate components that is mostly written in Go. While this enabled fast development, it lacks the memory safety provided by rust. Go is also fast during the runtime, but has a much bigger memory footprint than Rust in general.

Already some parts of zos are developed (or redeveloped) from scratch in rust to avoid leaks, and memory related issues this inclued:

• zinit the init process and the system process manager
• rfs the rust implementation of 0-fs and is now used with zos 3

Redesign ?

ZOS was designed to have separate small daemons that are concerned with managed a certain resource(s). for example, a storage daemon which manages the physical disks, manage access to it and allocation of storage resources. For example, creating a subvolume or a virtual disk is done via a certain call to the daemon that then find the best physical resource (disk) to fullfill this request.

The when another service needs to allocate a storage resource, it can make a call to storaged over the system bus zbus. ZBUS is an inter-process communication protocol that uses redis as a message bus, but unfortunately did not take portability into account but very easy to use from Go apps (client, and server).

So to move to Rust, we probably need to rewrite the separate "components" in rust, but then it's gonna be problematic to integrate with already existing go components.

So to overcome this we need first to rewrite zbus in a cross-language way. this can be done over stages as follows:

• use a different encoding than msgpack while this is supposedly cross compatible across languages, it's not as straight forward as it claims, since the 2 types has to respect the order of type attributes which is not granted across Go and Rust without manual work.
• ZBUS stubs generators, need to be do more:
  • Define service interface in Go
  • Define types in Go
  • Generate client code in Go lang (stubs)
  • Generate sesrver code in Rust from the Go interface
  • Generate client code in Rust from the Go interface

Can we just drop zbus entirely ?

Yes, but this will require the system to be re-written either completely in rust, or find a zbus replacement which will be even more work to move all current services to use