Pandoc WASM

[tarleb]

For those interested: @TerrorJack successfully compiled pandoc to WASM via ghc-wasm-meta.

• Livedemo: https://tweag.github.io/pandoc-wasm/
• Repo: https://github.com/tweag/pandoc-wasm

Fantastic work!

[fiapps]

It's too bad Lua filters don't work yet. This makes it less useful.

I experimented with it briefly using the iOS app a-Shell. I copied the URL of pandoc.wasm from the README and downloaded it with curl. When I run it, it hangs, but if I press CTRL-C, it produces the expected output. The hang isn't due to an attempt to read from stdin, because CTRL-D has no effect. Probably, it needs to be build with a-Shell's custom WASI-SDK, and perhaps some of TerrorJack's customization needs to be undone. Since this is not wasmtime but another platform, it may be that Lua filters can be made to work. However, I don't expect to dedicate time to this in the near future. Since I have no experience with Haskell, I imagine I'd have a hard time making a custom build of pandoc.

[jgm]

@TerrorJack this is amazing!

I'm wondering whether we should explore modifying the main pandoc repo with a wasm build configuration, so that a separate fork is not needed -- perhaps with CPP conditionals. Do you think this would be worthwhile?

[jgm]

If we do this I'd also like to provide pandoc.wasm in the Releases section.
That will make it quite easy for people to use.

[tarleb]

I'd be very interested to hear tips on how to get Lua working on wasm. Which steps would be necessary? Would we need to compile Lua separately and create a completely new interface with JS FFI to interface with it?

[TerrorJack]

Re lua support: I gave it a quick try today, and there are a couple of issues:

• lua makes use of some posix functions that are not present on wasi, the most troublesome one is tmpnam. This can be fixed with either assuming /tmp is present (and bringing in non portable assumption about the virtual fs is provided by the wasm runtime), or letting os_tmpname always bail out with error. I prefer the latter, though I don't know how large it might impact downstream users like pandoc filters.
• lua makes use of setjmp/longjmp. This is already supported by wasi-sdk by lowering to legacy eh opcodes, supported by chrome/firefox/safari, but it's not supported by wasmtime, so it hurts the portability of wasm module a bit.
• isocline does not work for wasm32-wasi and needs to be patched, you can't expect anything terminal related functionality unless you're rolling custom tty functionality outside of wasi standard using something like xterm.js

No real challenges here. I'm not including lua support in my initial pandoc.wasm just for simplicity, before we bring in lua we need to settle down on the tradeoffs, and then the implementation would not be hard.

[jgm]

This is encouraging. Because our build process already allows you to build pandoc without Lua support, we could provide both a portable wasm without Lua support and a less portable one with Lua support.

@tarleb would be able to say whether we ever use os_tmpname.

[tarleb]

I believe that os.tmpname is almost never used in third-party scripts, and definitely not in our scripts. The most common use-cases require a temporary directory, which can be generated via pandoc.system.with_temporary_directory, to that's what most people use. But even then, most filters that use tempfiles of any kind would require additional software and wouldn't work on wasm anyway, so it seems perfectly fine to break them and just ignore tempfiles.