core vs interface type imports/exports

[fgmccabe]

When crafting an adapter for an imported/exported function, it seems clumsy to have to require that any core wasm functions called in the adapter code have to be exported/imported. The adapter is written by the owner of the importing/exporting module and there they 'know' where all the functions are.
There is a case involving polyfill; but if polyfills are viewed as being implemented partly in JS and partly in wasm then they too do not need to export internal functions. (E.g., polyfill could invoke a serialization functionality within the wasm module and have the serial format interpreted/glued in JS.)

[lukewagner]

What requiring core functions/memories/etc be exported from the core module buys us is:

• spec layering: otherwise, the core spec would have to be changed to, effectively, export all definitions, and this would hurt the ecosystem's ability to, e.g., write analyses or optimizations, etc, that only understood core wasm concepts (treating only calls to imports as black boxes). In general, layering with tight interfaces between the layers (like we have now with imports/exports) is a pretty nice design ideal, I think.
• polyfilling: a useful property for polyfilling purposes is that one could polyfill at load-time and, at this stage, be able to polyfill by only probing for and reading the contents of custom sections (in order to generate glue code), but not otherwise mutate the .wasm binary.

I think the apparent clumsiness could be addressed by sugar/conventions in the toolchain (both source code and wat).

[fgmccabe]

I don't think I am proposing exporting everything. Just that the whole module is available to the adapter code. I am not super-strong in advocating this pov; it is just that when writing the adapter scenarios it seemed clumsy to go through the export mechanism when I 'knew' all the code anyway. BTW: What I have been thinking about polyfill would not involve modifying or reading the core wasm code. But polyfill might end up being a combination of JS and wasm generated code. There remains the issue where the interface type version of a functionality can look very different to the core wasm version. I believe that this is intrinsic; i.e., inescapable.

…

On Fri, Oct 11, 2019 at 11:46 AM Luke Wagner ***@***.***> wrote: What requiring core functions/memories/etc be exported from the core module buys us is: - spec layering: otherwise, the core spec would have to be changed to, effectively, export *all* definitions, and this would hurt the ecosystem's ability to, e.g., write analyses or optimizations, etc, that only understood core wasm concepts (treating only calls to imports as black boxes). In general, layering with tight interfaces between the layers (like we have now with imports/exports) is a pretty nice design ideal, I think. - polyfilling: a useful property for polyfilling purposes is that one could polyfill at load-time and, at this stage, be able to polyfill by only probing for and reading the contents of custom sections (in order to generate glue code), but not otherwise mutate the .wasm binary. I think the apparent clumsiness could be addressed by sugar/conventions in the toolchain (both source code and wat). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#77?email_source=notifications&email_token=AAQAXUBUSHFGKT22DOUY7XDQODCYNA5CNFSM4I7PVTAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBA4GQA#issuecomment-541180736>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAQAXUDV4FVH64V2BYAZEO3QODCYNANCNFSM4I7PVTAA> .

-- Francis McCabe SWE

[lukewagner]

I started to write out in more detail how we might add some desugaring rules to the @interface text format (similar to how (func (export "foo") ...) expands to both a (func ...) and (export ...) definition in core wasm text format), but I realized that, for both adapted imports and exports, both the adapted and core function signatures are meaningful b/c neither is automatically derived from the other.

Thus, the only sugar I could imagine is allowing:

(module
  (func $foo ...)
  (@interface func (export "foo")
     call $foo
  )
)

to be written and desugar into:

(module
  (func (export "foo_") ...)
  (@interface func (export "foo")
     call-export "foo_"
  )
)

The latter is definitely noisier, but it's not more lines of code, so I wonder if it's really worth it. Thoughts?

[fgmccabe]

I was already using the sugared form .. in my ignorance. There is more information in the second; I am not positive that information is necessary.

…

On Mon, Oct 14, 2019 at 3:01 PM Luke Wagner ***@***.***> wrote: I started to write out in more detail how we might add some desugaring rules to the @interface text format (similar to how (func (export "foo") ...) expands to both a (func ...) and (export ...) definition in core wasm text format), but I realized that, for both adapted imports and exports, both the adapted and core function signatures are meaningful b/c neither is automatically derived from the other. Thus, the only sugar I could imagine is allowing: (module (func $foo ...) ***@***.*** func (export "foo") call $foo ) ) to be written and desugar into: (module (func (export "foo_") ...) ***@***.*** func (export "foo") call-export "foo_" ) ) The latter is definitely noisier, but it's not more lines of code, so I wonder if it's really worth it. Thoughts? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#77?email_source=notifications&email_token=AAQAXUAFYGOFXZABECFZEWDQOTT4BA5CNFSM4I7PVTAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBGXOSQ#issuecomment-541947722>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAQAXUB4HNH5AQDS5E257Q3QOTT4BANCNFSM4I7PVTAA> .

-- Francis McCabe SWE

[jgravelle-google]

The discussion in #75 has got me thinking it might be better to use indices here and make it a semantic diff to core wasm. I'm thinking we should go as far as having this be a non-custom section.

My thoughts are: 1) I think that a runtime JS polyfill is more of a nice-to-have than a hard requirement. The more production-ready use case is going to be doing something like wasm-bindgen offline to pre-process it anyway. 2) it's potentially scary to break the encapsulation of a module by requiring a polyfill to fix up one's exports in order to maintain your boundary guarantees. If I don't want my memory exported, I'm trusting the polyfill to fix up after itself, and if I'm shipping a library I don't have control over the polyfill that gets used. 3) it makes it much easier to explain that the presence of the section has major semantic implications on the ABI, and that the section is non-ignorable by a standards-following engine.

[lukewagner]

Defining specs as diffs to other specs feels a bit like using comefrom; it seems like a sign of bad layering ;)

I think the non-ignorability of the adapter section is a natural consequence of the fact that the core module's interface is super-incompatible with the adapted module interface, so the user of an adapted module really doesn't have a choice but to apply the adapter to the core module.

Also, regarding polyfills, I think the polyfill technique won't just be a short-term or only-JS thing: because it'll be non-trivial to fully integrate interface types into the N different VMs (Python, Ruby, ...) that would want to call wasm, a reasonable technique in each of these languages is to use a wasm runtime to implement core wasm, and then do some glue-code-generation or adapter-function-interpreter to connect the wasm runtime with the containing VM. Wasm runtimes only expose a module's exports, so this lines up nicely.