Experimental Draft: Rkyv #1034
Experimental Draft: Rkyv #1034
Conversation
Introduces a new benchmark suite comparing rkyv and serde serialization/deserialization performance for Dynamic values, strings, and floats. Adds safe wrapper functions for deserializing Dynamic and generic types from bytes, and updates module exports to include these new functions.
Introduces `to_dynamic` and `from_dynamic` aliases for `to_bytes` and `from_bytes_owned` to provide API compatibility with serde. Documentation is updated to clarify the differences and usage for users familiar with serde.
This commit extends rkyv serialization/deserialization to support arrays and maps by recursively archiving nested Dynamic values. It updates SimpleDynamic to include Array and Map variants, adds helper functions for nested serialization, and provides new examples and tests for arrays, nested arrays, and maps with nested structures. Documentation is updated to reflect supported types and recursive behavior.
| categories = ["no-std", "embedded", "wasm", "parser-implementations"] | ||
|
|
||
| [dependencies] | ||
| smallvec = { version = "1.7.0", default-features = false, features = ["union", "const_new", "const_generics"] } |
There was a problem hiding this comment.
See if you can avoid generating many changes due to auto-reformatting. This distracts from the actual changes in the file.
| @@ -0,0 +1,143 @@ | |||
| #![feature(test)] | |||
There was a problem hiding this comment.
Benches are run upon commit in my dev repo, so I'm not sure if we'll blow up GitHub by adding so many new benches. We can try.
The benchmarks are not run in the official repo which is named main. I named mine master which will trigger running benchmarks and collecting performance data upon push.
| //! * Script caching - Load compiled scripts 50-100x faster | ||
| //! * State snapshots - Save/restore engine state with minimal overhead |
There was a problem hiding this comment.
I don't think rkyv will help in serializing a compiled AST or Engine run state here...
At least not at this point as we're only serializing Dynamic data.
Actually this is an interesting idea... Can we serialize an AST such that it is zerocopy and can be passed directly into the Engine without recreating a full AST structure again?
I'm not convinced this'll work but again, who am I to say it definitely cannot.
There is only value is the serialized AST can be used directly from the serialized bytes. If recreating the AST structure is needed, then reparsing the script text may not be too slow compared to a deserislization step.
There was a problem hiding this comment.
Technically i think yes we can. But i am not sure how much benefits we can get over the complexity as we might have to implement Archieve trait to all AST types and i am finding hard time to make rkyv work with complex recursive structures. This is just my over the top answer as i havent gone in depth in this one. But i definately want to. Will keep you posted regarding this.
|
@schungx Thank you for the comments. Overall i would say this is gonna be a experiemental stuff for me and if i can land something to make it work. I will clean and make the diff comparable so its easier to track only the changes i have made. Thank you again. |
Eliminated all println! debug statements from SimpleDynamic conversion logic for cleaner code. Enhanced handling of Union::Variant by adding explicit downcasting to common types before falling back to Dynamic API conversions.
2 participants
#1033