Minimize buffering w/ v1.1 Binary writer?

[barries]

I've been digging through the 1.1 rc10 reference docs to figure out how to configure it to minimized buffering--for example by using delimited sequences. I'm afraid I haven't figured it out, so... how can I configure the v1.1 writer to minimize buffering and other overhead when using both the imperative and Serde serialization APIs?

Thanks!

For context, the prototype so far is:

    let writer = IonWriter::new(
        v1_1::Binary,
        File::create(log_file_path.clone())?,
    ).expect("BinaryWriter::new() failed");
    ... store writer in self.ion_writer ....

   
    pub fn write_entry(
        &mut self,
        module_path:   &str,
        event_kind:    EventKind,
        write_content: impl FnOnce(&mut Log),
    ) {
        {
            use chrono::Utc;

            let now = Utc::now().naive_utc();
            let event_timestamp = ion_rs::Timestamp::from_datetime(now, ion_rs::TimestampPrecision::Second);
            let _ = self.ion_writer.write(event_timestamp);
            let _ = self.ion_writer.write_symbol(format!("{:?}", event_kind).as_str());
            write_content(self);    // <<== usually a closure that calls write_value(...)
        }
        let _ = self.ion_writer.flush();
    }

    pub fn write_value(&mut self, value: &impl serde::Serialize) {
        use ion_rs::{
            SequenceWriter,
            serde::ser::ValueSerializer,
        };

        let is_human_readable = false;
        let serializer = ValueSerializer::new(
            self.ion_writer.value_writer(),
            is_human_readable
        );
        let _ = value.serialize(serializer);
    }

[zslayton]

You can call value_writer().with_delimited_containers() to skip buffering for containers. Setting that also sets it for any nested containers (unless you set it back for those again).

Here's the type that holds all of the currently available options:

ion-rust/src/lazy/encoder/value_writer_config.rs

Lines 1 to 15 in 4e8739f

	/// Configuration options available to Ion 1.1 value writers.
	///
	/// The default configuration aligns closely with Ion 1.0's encoding. All symbols, field names, and
	/// annotations are added to the symbol table and encoded as symbol IDs.
	#[derive(Copy, Clone, Debug, Default)]
	pub struct ValueWriterConfig {
	// How nested containers should be encoded.
	container_encoding: ContainerEncoding,
	// How symbol values should be encoded.
	symbol_value_encoding: SymbolValueEncoding,
	// How annotation sequences should be encoded
	annotations_encoding: AnnotationsEncoding,
	// If this writer emits a struct, the struct will encode its field names according to this setting.
	field_name_encoding: FieldNameEncoding,
	}

Each of those enums are defined just below it. Let me know if I can answer more questions.

[barries]

Thanks, @zslayton.

That's not present on ApplicationValueWriter<>, should it be, or am I using the wrong API somehow?

type IonWriter = Writer<v1_1::Binary, File>;

pub struct Log {
    ion_writer: IonWriter,
}
...
error[E0599]: no method named `with_delimited_containers` found for struct `ion_rs::lazy::encoder::writer::ApplicationValueWriter` in the current scope                                                                                        --> src/uilog.rs:418:67
    |
418 |             let mut entry_writer = self.ion_writer.value_writer().with_delimited_containers();
    |                                                                   ^^^^^^^^^^^^^^^^^^^^^^^^^ method not found in `ApplicationValueWriter<'_, BinaryValueWriter_1_1<'_, '_>>`

[zslayton]

Hmm, it should be there. The source code for those methods is here:

ion-rust/src/lazy/encoder/writer.rs

Lines 243 to 267 in df7ba6c

	impl ApplicationValueWriter<'_, BinaryValueWriter_1_1<'_, '_>> {
	pub fn with_container_encoding(mut self, container_encoding: ContainerEncoding) -> Self {
	self.value_writer_config = self
	.value_writer_config
	.with_container_encoding(container_encoding);
	self
	}
	pub fn with_annotations_encoding(mut self, annotations_encoding: AnnotationsEncoding) -> Self {
	self.value_writer_config = self
	.value_writer_config
	.with_annotations_encoding(annotations_encoding);
	self
	}
	pub fn with_symbol_value_encoding(
	mut self,
	symbol_value_encoding: SymbolValueEncoding,
	) -> Self {
	self.value_writer_config = self
	.value_writer_config
	.with_symbol_value_encoding(symbol_value_encoding);
	self
	}
	}

Note that it's only implemented for ApplicationValueWriter<'_, BinaryValueWriter_1_1<'_, '_>> since binary 1.1 is the only encoding that has opt-in delimited containers.

[barries]

When I click on that link, I see this

When I look in my ion-rust main branch a few commits after .rc10, I only find it here:

impl BinaryValueWriter_1_1<'_, '_> {
    ...
    
    pub fn with_delimited_containers(mut self) -> Self {
        self.value_writer_config = self.value_writer_config.with_delimited_containers();
        self
    }

What am I doing wrong?