Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question, what's the intension of libc-bottom-half? #216

Closed
lygstate opened this issue Sep 15, 2020 · 12 comments
Closed

Question, what's the intension of libc-bottom-half? #216

lygstate opened this issue Sep 15, 2020 · 12 comments

Comments

@lygstate
Copy link

Why not just direct lowring down from musl to wasm syscall?

@sunfishcode
Copy link
Member

musl assumes it's running on top of a Linux system call API. WASI is different from Linux system calls in several ways:

  • It's capability-oriented, so eg. filesystem paths are always relative to base, rather than being absolute.
  • It's significantly simpler. Linux has a fair number of obsolete, redundant, rarely used, and overly complex system calls, and WASI is aiming to be a simple and clean API layer.

The "bottom half" of musl contains infrastructure for performing system calls, however in WASI, system calls can be called like normal functions and don't need that infrastructure. So we disable musl's "bottom half" and replace it a WASI-aware bottom half.

@lygstate
Copy link
Author

musl assumes it's running on top of a Linux system call API. WASI is different from Linux system calls in several ways:

  • It's capability-oriented, so eg. filesystem paths are always relative to base, rather than being absolute.
  • It's significantly simpler. Linux has a fair number of obsolete, redundant, rarely used, and overly complex system calls, and WASI is aiming to be a simple and clean API layer.

The "bottom half" of musl contains infrastructure for performing system calls, however in WASI, system calls can be called like normal functions and don't need that infrastructure. So we disable musl's "bottom half" and replace it a WASI-aware bottom half.

What's the reason to not place bottom half inside musl? and merge these changes upstream to musl main repository?
Currently we have a whole copy musl and we can not distinguish the differencies betwen wasl-lib vs musl.

@sbc100
Copy link
Member

sbc100 commented Sep 16, 2020

My understanding is that upstream musl is not interested in porting to non-linux platforms so would not accept such changes (at least that was that case several years ago).

@lygstate
Copy link
Author

My understanding is that upstream musl is not interested in porting to non-linux platforms so would not accept such changes (at least that was that case several years ago).

We don't have to upstream everything, A minamal set that easy to create conflict upstreamed are enough. And also makes upstream easy to rebase on upstream musl.

@sunfishcode
Copy link
Member

I agree; we've already upstreamed several bug fixes, and will continue to upstream where it makes sense. As @sbc100 says, a lot of our changes that we haven't upstreamed are things we don't expect upstream musl would be interested, at least as things stand right now.

@sunfishcode
Copy link
Member

Wrt having all the musl code, it would be feasible to actually delete the parts of musl we aren't using, to make it easier for people to browse. I unfortunately don't have time to do that at the moment, but I could provide pointers if someone wanted to work on that.

@lygstate
Copy link
Author

Wrt having all the musl code, it would be feasible to actually delete the parts of musl we aren't using, to make it easier for people to browse. I unfortunately don't have time to do that at the moment, but I could provide pointers if someone wanted to work on that.

What stop wasi-libc to implement these things? May use create a expremential option and upgrade it latter?
These four feature are very important to create a usable wasi-sdk

This repository does not yet support C++ exceptions. C++ code is supported only with -fno-exceptions for now. Similarly, there is not yet support for setjmp/longjmp. Work on support for exception handling s underway at the language level which will support both of these features.

This repository does not yet support threads. Specifically, WASI does not yet have an API for creating and managing threads yet, and WASI libc does not yet have pthread support.

This repository does not yet support dynamic libraries. While there are some efforts to design a system for dynamic libraries in wasm, it is still in development and not yet generally usable.

There is no support for networking. It is a goal of WASI to support networking in the future though.

@lygstate
Copy link
Author

Wrt having all the musl code, it would be feasible to actually delete the parts of musl we aren't using, to make it easier for people to browse. I unfortunately don't have time to do that at the moment, but I could provide pointers if someone wanted to work on that.

The upstream libc-bottom-half cloudlibc are already dead, how about merge it into musl?

@sunfishcode
Copy link
Member

A lot of it comes down to work that needs to be done. Most of what you see here in WASI libc is done by a very small number of people. Much more would be possible if we had more people helping.

C++ exceptions

WebAssembly doesn't currently have a way to unwind the stack. To implement exceptions today, we'd either need to teach the LLVM backend to lower exception handling into extra return values and branches, or we'd need an API to call into and get called back in from the host (which is fairly complex). If someone wants to volunteer to do this work, these could theoretically happen. However, in the future, real exception handling is coming, and will obviate anything we do now. Since we have such limited resources, we haven't prioritized work on the temporary solutions.

threads

This is a similar situation. The WebAssembly threads proposal doesn't support non-Web threads well. See WebAssembly/WASI#296 (comment) for details. A lot of people are interested in having something that works, but so far no one has done the work.

dynamic libraries

The plan for dynamic linking is here: https://github.com/WebAssembly/module-linking/

If you want duct-tape solutions, other tools are available, but we're trying to build a real platform and not just get things to work via the fastest path.

networking

The spec work is underway. The limiting factor is human bandwidth.

@sunfishcode
Copy link
Member

The upstream libc-bottom-half cloudlibc are already dead, how about merge it into musl?

WASI is significantly different than Linux syscalls. And, it's still evolving. Upstreaming this code into musl right now would make it slower for us to do the evolution to add the features you're asking for, and would create a lot of churn for musl upstream. As the person who has done several rebases of WASI libc onto newer musl revisions, please understand that I'm well aware of the costs, and I pay them because blindly upstreaming everything would have greater costs.

@sunfishcode
Copy link
Member

If you want duct-tape solutions, other tools are available, but we're trying to build a real platform and not just get things to work via the fastest path.

To provide additional context for this remark, the other tools that do this aren't necessarily duct-tape within their domains. For example, in a Web context where one person publishes all the libraries themselves, it's not critical to have a stable dynamic library ABI, because one can simply build everything with the same version of the tools. However for other uses, dynamic libraries would more often be used in settings where stable ABIs are required, so more work is needed to ensure that we have a base design that we can grow to support the features we'll want to add as wasm evolves.

@sunfishcode
Copy link
Member

The main questions here appear answered; please re-open or file new issues if there are further questions!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants