Use LFortran to compile any Fortran to WASM

[certik]

LFortran (as of today) can compile 6 out of 18 Fortran packages in SciPy: lfortran/lfortran#2743, and we keep working on compiling the rest. At the same time, I think it's a good time to start using LFortran to compile SciPy to WASM, since it will take some time to iron out all issues. LFortran is available in conda-forge, works on all platforms, etc. We would like to make LFortran work seamlessly as a robust tool that you can depend on and that will eventually remove all pain issues with the tooling around Fortran.

We can start with SciPy (unless you have a better package to try). Here is my understanding so far:

• The initial support was added in add scipy #123, and it used the compiler wrapper https://github.com/wolfv/emscripten-forge/blob/4bd3c62f2d97184126615162783b32337afe1b23/recipes/recipes_emscripten/scipy/patches/fortran_compiler_wrapper.py, which uses f2c and emcc to compile old Fortran. It contains a lot of hacks. LFortran could replace all this by compiling Fortran to WASM directly (via LLVM). We have various compiler options to control name mangling that we use to make all SciPy tests pass (via LLVM -> native code, we have not tried WASM yet).
• The wrapper was later removed in Emscripten 3.1.45 and Python 3.11 update #643 and replaced with the binary https://github.com/DerThorsten/recipes/blob/b6227f49deeaa8f3a2455451fd294405156a3510/recipes/recipes_emscripten/scipy/scipy-1.11.1-cp311-cp311-emscripten_3_1_45_wasm32.whl. So far I have not figured out how the whl is built.

@martinRenou, @DerThorsten how can we help? This will be a long term iterative process, but we want to get started. The nice thing is that once we get SciPy working, LFortran will (almost) automatically work for any other Fortran package out there, so this is a great long term investment.

[DerThorsten]

Atm we take the whl build from pyodide https://github.com/pyodide/pyodide/tree/main/packages/scipy

We did that because I was too lazy to update all the above-mentioned hacks when updating to an new scipy version (these hacks are all still done by pyodide, and our initial build script was adapted from pyodide by @wolfv)

[wolfv]

I would be super happy to test doing this with you in the emscripten-forge context, @certik. Personally I prefer building scipy from source here instead of taking it from pyodide.

[certik]

@wolfv, @DerThorsten awesome. Let's say LFortran can generate the same wasm output as f2c+emcc. How do you then call it from Python? Do you use the f2py somehow, like scipy does? I was thinking of testing this using some small demo, to ensure everything works.

[wolfv]

Emscripten can load the shared objects just fine iiuc

[certik]

Ah, so we compile the Fortran code to an "object file" that emcc then loads when it is being linked to the main application? I will need to setup a working example first.

[wolfv]

That would be an interesting start. I think one can also dlopen a SO file with emscripten. I am not sure if emscripten needs anything special.

Maybe a good start could be to take the existing scipy package, create the WASM environment with micromamba, and then exchange the .so files in there?

[DerThorsten]

@certik I added instructions how to setup local builds with emscripten forge https://github.com/emscripten-forge/recipes#local-builds

The easiest way to build a new python package and test it is probably via a new recipe.
The build does not necessarily need happen inside the recipe.
Currently, we just take some externally built wheel in the scipy recipe and extract it.
But doing so would allow us to use all the mechanisms we have to test the packages via playwright in a headless browser