Bundle mainnet genesis state

[grod220]

References #161, pairs with penumbra-zone/web#1706

[changeset-bot]

⚠️ No Changeset found

Latest commit: dfa99dc

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[TalDerei]

closing #163 in favor of this.

[grod220]

@TalDerei feel free to take this over as it pairs with your other work

[hdevalence]

Looking at the rust code here https://github.com/penumbra-zone/web/blob/main/packages/wasm/crate/src/view_server.rs#L116 , isn't the compact block passed into wasm as a binary-proto-encoded byte array? in that case, wouldn't it be best to store the compact block in that format to avoid serialization overhead, and which would also be smaller to include in the extension manifest?

[TalDerei]

@TalDerei feel free to take this over as it pairs with your other work

thanks taking over PR 👍

[TalDerei]

Looking at the rust code here https://github.com/penumbra-zone/web/blob/main/packages/wasm/crate/src/view_server.rs#L116 , isn't the compact block passed into wasm as a binary-proto-encoded byte array? in that case, wouldn't it be best to store the compact block in that format to avoid serialization overhead, and which would also be smaller to include in the extension manifest?

In theory, storing the compact block object as a binary-proto-encoded byte array should be more efficient in terms of both serialization and space costs.

In terms of serialization, we'll still need to perform a deserialization operation in the extension when fetching the genesis file, CompactBlock.fromBinary(new Uint8Array(binaryData)), since the block processor processes and perform checks on the in-memory compact block object before converting it to the byte array that wasm consumes. However, deserializing from binary should be more efficient than deserializing from JSON.

In terms of space, the compact block object for the genesis block is ~10 MB, while its JSON representation is ~18 MB. Surprisingly, when serialized to a binary-proto-encoded byte array using compactBlock.toBinary() and saved to a binary file, it takes up ~150 MB.

cc @turbocrime

[hdevalence]

I'm not sure where those numbers are coming from. The genesis compact block is 8035648 bytes. See penumbra-zone/penumbra#4824 for code to generate the data

[TalDerei]

I'm not sure where those numbers are coming from. The genesis compact block is 8035648 bytes. See penumbra-zone/penumbra#4824 for code to generate the data

strange blowup, I was simply logging the binary representation of the compact block and manually saving to a file.

[grod220]

Tested this out locally and things are working for me. @TalDerei after publishing the latest packages in penumbra-zone and pulling them in here, think you can merge.

In general, it's nice this saves the 8mb download. However, for a full sync the user will still have to download another +60mb. Connectrpc downloads all of those in parallel to the block sync logic. That said, it's not really noticed on my machine as the CPU/IO tasks are the bottleneck and those downloaded blocks are just waiting to be processed.

[TalDerei]

That said, it's not really noticed on my machine as the CPU/IO tasks are the bottleneck and those downloaded blocks are just waiting to be processed.

echoing the same observation on M3 pro

[TalDerei]

fetch api goes through the browser's network stack for local files, and there seems to be overhead in setting up the request, but latency is negligible