3_6_serde

Step 3.6: Serialization and deserialization

Estimated time: 1 day

serde

Rust ecosystem has the well-known serde crate, which provides a common (standard, de facto) approach and toolset for serialization and deserialization.

The sweet part is that serde does not rely on a runtime reflection mechanism and uses trait implementation for each type, so eliminates most runtime costs and in most cases makes serialization as performant as handwritten serializer for a particular case, yet remains ergonomic due to automatic code deriving.

use serde::{Deserialize, Serialize};

#[derive(Debug, Deserialize, Serialize)]
struct Point {
    x: i32,
    y: i32,
}

fn main() {
    let point = Point { x: 1, y: 2 };

    let serialized = serde_json::to_string(&point).unwrap();
    println!("serialized = {}", serialized);

    let deserialized: Point = serde_json::from_str(&serialized).unwrap();
    println!("deserialized = {:?}", deserialized);
}

serde by itself represents only a universal serialization frontend, which can be backed by actual implementation for any format. There are already implemented backends for most used formats, and you're free to implement backend for your own format if it's not implemented yet.

For better understanding and familiarity with serde's design, concepts, usage, and features (like zero-copy deserialization), read through the following articles:

• Official serde crate guide
• Official serde crate docs
• Official serde_json crate docs
• Owen Gage: Understanding Rust's serde using macro expansion
• Owen Gage: Exploring serde's data model with a toy deserializer
• Owen Gage: A look at serde-json
• Manish Goregaokar: Not a Yoking Matter (Zero-Copy #1)
• Manish Goregaokar: Zero-Copy All the Things! (Zero-Copy #2)
• Manish Goregaokar: So Zero It's ... Negative? (Zero-Copy #3)

Extras

Being the de facto ecosystem standard, serde crate itself is quite conservative about stability guarantees, so often may feel lacking obvious features. Therefore, additional ecosystem crates are worth considering, which extend serde capabilities, being built on top of its machinery:

• erased-serde crate, providing type-erased versions of serde’s Serialize, Serializer and Deserializer traits that can be used as trait objects.
• serde_state crate, extending the normal Deserialize and Serialize traits to allow state to be passed to every value which is serialized or deserialized.
• serde_repr crate, deriving serde's Serialize and Deserialize traits in a way that delegates to the underlying repr of a C-like enum.
• serde_with crate, providing custom de/serialization helpers to use in combination with serde’s with-annotation and with the improved serde_as-annotation.
• serde_valid crate, enabling JSON Schema based validation.

musli

musli is a relatively fresh and alternative framework for serialization and deserialization, which succeeds the principles of serde, but also rethinks and overcomes some of its fundamental limitations.

Müsli is designed on similar principles as serde. Relying on Rust’s powerful trait system to generate code which can largely be optimized away. The end result should be very similar to handwritten highly optimized code.

Where Müsli differs in design philosophy is twofold:

We make use of GATs to provide tighter abstractions, which should be easier for Rust to optimize.

We make less use of the Visitor pattern in certain instances where it’s deemed unnecessary, such as when decoding collections. The result is usually cleaner decode implementations

However, the main "killer feature" of musli is its ability to serialize/deserialize the same data model in different modes.

Another major aspect where Müsli differs is in the concept of modes (note the M parameter above). Since this is a parameter of the Encode and Decode traits it allows for the same data model to be serialized in many different ways.

use musli::mode::{DefaultMode, Mode};
use musli::{Decode, Encode};
use musli_json::Encoding;

enum Alt {}
impl Mode for Alt {}

#[derive(Decode, Encode)]
#[musli(mode = Alt, packed)]
#[musli(default_field_name = "name")]
struct Word<'a> {
    text: &'a str,
    teineigo: bool,
}

let CONFIG: Encoding<DefaultMode> = Encoding::new();
let ALT_CONFIG: Encoding<Alt> = Encoding::new();

let word = Word {
    text: "あります",
    teineigo: true,
};

let out = CONFIG.to_string(&word)?;
assert_eq!(out, r#"{"text":"あります","teineigo":true}"#);

let out = ALT_CONFIG.to_string(&word)?;
assert_eq!(out, r#"["あります",true]"#);

For better understanding and familiarity with musli's design, concepts, usage, and features, read through the following articles:

• Official musli crate docs
• John-John Tedro: A fresh look on incremental zero copy serialization

rkyv

rkyv (archive) is an another alternative serialization/deserialization framework, fully focused on zero-copy operations.

Like serde, rkyv uses Rust’s powerful trait system to serialize data without the need for reflection. Despite having a wide array of features, you also only pay for what you use. If your data checks out, the serialization process can be as simple as a memcpy! Like serde, this allows rkyv to perform at speeds similar to handwritten serializers.

Unlike serde, rkyv produces data that is guaranteed deserialization free. If you wrote your data to disk, you can just mmap your file into memory, cast a pointer, and your data is ready to use. This makes it ideal for high-performance and IO-bound applications.

While rkyv is a great format for final data, it lacks a full schema system and isn’t well equipped for data migration and schema upgrades. If your use case requires these capabilities, you may need additional libraries the build these features on top of rkyv. You can use other serialization frameworks like serde with the same types as rkyv conflict-free.

For better understanding and familiarity with rkyv's design, concepts, usage, and features, read through the following articles:

• Official rkyv crate docs
• rkyv book

Task

Write a program which deserializes the following JSON into a static Request type and prints out its serialization in a YAML and TOML formats. Consider to choose correct types for data representation.

Prove your implementation correctness with tests.

Questions

After completing everything above, you should be able to answer (and understand why) the following questions:

• How does serde achieve its performance? How does it model data and decouple responsibilities?
• When does it have sense to prefer musli rather than serde?
• What is zero-copy deserialization? Why is it beneficial? How does it work in serde? How does it work in rkyv?