Module splitting and type section duplication

[tlively]

Hello! I've been looking into more robust module splitting solutions, particularly for WasmGC modules. Here are the results of an experiment I did where I split calcworker_wasm.wasm from Google Sheets into 216 modules. Functions were automatically assigned to modules based on their original build targets rather than what would make sense to actually serve, but this should suffice to get an idea of the overheads involved.

First, here's how overall code size was affected, along with a breakdown by section of where the change came from.

module	file size	type	code	global	import	export
original calcworker_wasm.wasm	3702274	84324	2507784	508017	275383	210
cumulative split modules	5430287	1063595	2597670	506189	587786	277331
percent increase	46.67%	1161.32%	3.58%	-0.36%	113.44%	131962.38%
share of blame		56.67%	5.20%	-0.11%	18.08%	16.04%

We can probably reduce the code size due to imports and exports a bit more by moving more than just functions into the secondary modules. For example, if a secondary module is the only one that uses a particular imported global, it could just import the global directly. Today, the primary module imports and re-exports the global, then the secondary module imports it from the primary module.

But there's nothing we can do today to improve the code size of the type sections! The types are already arranged into minimal recursion groups and included only in modules where they are necessary for validation. Here's a breakdown of how many types each module uses either directly or indirectly. A directly used type is one that is in a rec group with a type that is directly allocated, accessed, cast, or otherwise referenced from the code. All other types are indirectly used, and are necessary to include only because they appear somewhere in the expanded definition of a directly used type.

module	included types	directly used types	percent used
original calcworker_wasm.wasm	5692	5686	99.89%
cumulative split modules	56825	31181	54.87%
multiplicative factor	9.98	5.48	0.55

On average, each type appears in about 10 modules, but is only directly used in 5.5 of them.

I'm interested in hearing what ideas folks have about how we could reduce the overhead of duplicated type sections. The best case would be that we could directly use the full type section from the primary module in each of the secondary modules without having to download it again. Another solution might look more like compile-time type imports that are able to abstract away the unused types, but that would still require repeating the used types. Either way, I don't have all the details worked out. Are there other or more complete ideas out there?

---

For completeness, here's how the code bloat looks when the modules are compressed.

module	file size	gzip size	brotli size	brotli -D main.wasm size
original calcworker_wasm.wasm	3702274	1106420	829443	829443
cumulative split modules	5430287	1778787	1457862	1110174
delta	46.67%	60.77%	75.76%	33.85%

Unsurprisingly, brotli compresses better than gzip, but because the baseline compression is better, the overhead from splitting is relatively worse. Using the primary module as a dictionary for the compression gives the best absolute and relative overheads. (Chrome supports shared dictionary compression as of a few weeks ago.)

[conrad-watt]

To make a brief comment here summarising my suggestion in the meeting - an extension to compileStreaming that takes an existing compiled Wasm module as an additional parameter, and compiles a new module which has the definitions of the existing compiled module as a prefix and the streamed bytes as the suffix. This would require a concept of "Wasm module fragments" and (if we want the suffix to extend both the types and imports sections) a concept of repeated sections.

If we just want the (complete) types section shared and nothing else, we could technically get away without repeated sections, just with a restricted concept of module fragments which are all the module sections after the type section (and at this point we're close to the raw "concatenate bytes instead" point that @rossberg made).

My arguments in favour of a "module fragment" approach rather than a byte approach would be:

• it seems clearly preferable if multiple sections (e.g. types and imports) need to be extended - the analogous stream slicing API with the byte approach seems pretty nasty
• in the fullness of time, engines might be able to reuse compilation work already done for the prefix module, which would have to be redone in the byte approach
• we want repeated sections for other reasons - if we had them already, I think an extension allowing them to be split across separate files would be obvious

To acknowledge some arguments against:

• it requires a new concept of "module fragment" in the ecosystem
• you can get a lot done with just naive byte concatenation (although see my perspective above)
• it's not a silver bullet for splitting the code section (deferred-loaded functions still likely need to be indirectly called, as the indexes of directly-called functions are likely too fragile among other ergonomic problems)
• if engines can't get all the necessary suffix compilation info from the compiled prefix module, they may need to keep additional info from the prior compilation live for longer (in the worst case, the full raw pre-compilation bytes)
• there are still open design questions about any kind of repeated section, let alone this extension

[conrad-watt]

If we just want the (complete) types section shared and nothing else, we could technically get away without repeated sections, just with a restricted concept of module fragments which are all the module sections after the type section (and at this point we're close to the raw "concatenate bytes instead" point that @rossberg made).

To expand on this point, we could imagine a limited version of my API where the provided prefix module must have all its sections after the type section empty, and the streamed "module fragment" must declare exactly the sections after the type section in regular module order. If we knew the general solution with full repeated sections was on our roadmap, this might be an acceptable MVP. However if we never planned to extend to the general thing, I think this solution would appear quite messy on its own (and would have questionable value over more naive byte-based or compression-based solutions).

[titzer]

Sorry I missed the meeting, the notes aren't up yet, so I don't know all the discussion points.

I echo the points @conrad-watt raised, to the extent I understand them without context. We should allow repeated sections; it's high time for this relaxation.

Module fragments as a more structured concept sounds more attractive than byte-level concatenation. I've had offline conversation with @rossberg about "modules with holes", which, while not entirely baked, could allow an outer module to be filled in with one or more sections or function bodies. It seems like submodule granularity is something worth reasoning through.

[rossberg]

@titzer, to clarify, module "fragments" would very much imply byte-level concatenation as well. A fragment as discussed in the meeting has no semantic meaning, it's just a piece of a module's binary representation that would be stitched together into an actual binary on the fly by the API. I agree that isn't very attractive. In the meeting I dared calling it a hack.

My point in the meeting was that the sections relaxation isn't strictly needed for this, at least not for the type section use case prompting the discussion, though it would obviously make the stitching more flexible.

That said, I'm very much in favour of allowing repeated sections in general.

[rossberg]

As for a more structured solution, I suggested experimenting with type imports to prune the type definitions not relevant to each module itself. I could imagine this might help significantly, since the underlying problem is that we currently have to include the transitive closure of all type definitions used by a module. And the size of that tends to grow combinatorially with the depth of the dependency graph.

[tlively]

I plan to investigate the type imports approach first. Although the module concatenation solutions would solve the code size problem perfectly, we have enough problems as it is integrating Wasm into JS build and serving infrastructure, so I would like to avoid introducing a new “module fragment” entity into the ecosystem if possible.

[tlively]

When we discussed this in the CG, there were some questions about compression. I've now updated the original post with comparisons of compressed code size, including when using the primary split module as a compression dictionary for the other split modules.