toward zero-copy marshal using capnproto

[dckc]

In a rambling discussion with @erights , we were talking about medium/long term performance approaches. I brought up...

• perf investigation: KREAd, ERTP, NFT Purses, Zoe Agoric/agoric-sdk#8862

where we find

There are a tremendous number of passStyleOf() calls in this portion of the flamegraph.

I asked @erights if he sees any path to zero-copy marshal, and he said perhaps if we used capnproto, combined with something like my protobuf schema for passable.
zenhack sketched ocapn.capnp; it's probably pretty close.

capnproto support for JavaScript doesn't get anywhere near the sort of optimized proxy @erights and I discussed; the library by @kentonv is SLOW (his words).
OCaml support looks quite good; the mirage folks seem to use it in anger. OCaml is a good match for the way I think.
But capnp-ocaml has very little in the way of "getting started" material.

I suspect nix could help put all the tools together, but it also adds complexity.
I'm kinda stuck. Or at least out of mental energy for now.

• capnproto ocapn flailing in ocaml (WIP) playground#9

[kentonv]

Realistically I wouldn't reach for Cap'n Proto as a way to solve performance problems in an existing system unless the use case specifically calls for zero-copy. For example, if you want to be able to mmap() a huge structure from disk and read only a small subset of it, that's where zero-copy will really shine. But if you are passing messages around for an RPC-ish use case, it won't necessarily help, and the work involved in changing serialization formats is probably quite large.

[dckc]

Thanks for chiming in, @kentonv . Yes, that's why this is a discussion and not an issue; it's a weekend thought experiment that I figured I would share, even if we aren't likely to pursue it seriously.

[mhofman]

This is a discussion we've been having on and off for a while. More recently related to #1478

IMO the problem is not really what format/codec we use to serialize / deserialize capdata, but how it's represented in our system across boundaries. In particular, we often need to nest CapData inside other messages, which are themselves serialized / deserialized.

So far our approach is that our messages don't have a specific / predefined schema, and that CapData does not have a special / recognizable type. As such the CapData needs to follow the serialization logic of wherever it gets nested, which for JSON means stringifying or parsing it eagerly.

In agoric-sdk, there are really 2 places where CapData appears:

• "supervisor level" in the payload of a liveslots delivery / syscall
• "user level" as the encoding of some external data, such as a smart wallet action, or vstorage data.

I don't believe true zero-copy is feasible in JavaScript. A byte representation will always need to be decoded into JS values, whether at the JS layer, or at the "network" layer. A JS Proxy wouldn't enable partial parsing because we aggressively harden objects. In the other direction, you would either need marshal to produce bytes and have the nesting layers pass through the bytes as-is, or have marshal avoid any serialization and rely on the network layer to serialize.

Throughout the stack, we are often interested in partial parsing CapData across boundaries: leave the body fully or partially unparsed, but parse the slots, e.g. for clist rewrite.

The supervisor level could have a predefined schema for all its messages, which would allow partial parsing of the CapData if needed.
With @gibson042 we discussed a way to declare that Far object / promise watcher handles "raw deliveries", which means it receives a CapData payload with the unparsed body but with slots replaced by the actual objects. But I'm still unsure if this solves anything.

The problematic use case is as follow: agoric-sdk vstorage values are often encoded as CapData. These values are currently marshalled by the board vat, which receives a toCapData call and uses its own marshaller to encode the payload in the argument. The CapData "copy-record" is sent back as result. On call resolution, the result is JSON stringified, before being sent as the argument of a storage node call. The node receives the setValue call (in the bridge vat), which in turn wraps the string argument (serialized capdata) in a array with the node's path, and uses it as argument to a device syscall. Everytime these values are used as arguments or results, they are included in a CapData encoded structure created by the liveslots supervisor, which is then wrapped in some JSON object.

< insert graph here with payload for every step>

As seen above, simply "handling raw deliveries" doesn't help by itself, since each delivery payload needs to be partially wrapped / unwrapped.
Even if we assume the board marshaller API would be refactored to not receive a full payload, but only the "slots" (there is really no reason for it to marshal the body), the CapData record that will ultimately be written to vstorage still needs to be encoded when passed around. Same if the board was changed to forward the result directly to the storage node (which might be a capability leak).

I'm honestly unsure how to solve these layering constraints besides having a first class notion of "serializable data" (hardened JS value which can be JSON stringified, maybe extended to support binary blobs), that passStyleOf would be able to recognize without traversing (to avoid eager parsing), and that can be passed around until it reaches a platform API which would take care of the serialization when sending over the wire. Marshal would need to produce a body that is not pre-serialized (generates a serializable body). Such serializable data would either be explicitly created using a platform API (explicit tag), or it would be an implicit emergent property of the hardened data (based on its content). For serialized data parsed by the platform, it could create a proxy-like object that identifies as serializable data (to avoid passStyle traversal) which lazy parses the data, and more importantly, is able to be "re-serialized" by simply copying the serialized data that was received and never parsed.

At that point we could imagine changing the vstorage API so that values could be serializable data instead of just strings (with the pending question of what to do with values already written that are raw string values instead of a JSON stringification).