[stdlib] Add Libc and tests

[martinvuyk]

Add Libc and tests

This will pave the way for better cross platform support.

Tests for the most used libc functionality were added. Some trivial functions were left as # TODO for later testing. ioctl was deactivated as it needs to be tested thoroughly.

On the networking side, I have functional socket code but it needs a counterpart to be used or use async code, so I left it "untested" but it already works. The next stage after this would be to bring in the socket package I've been working on for the last couple of months, but I'd rather wait for full async Mojo since many APIs would benefit from being async first.

I can keep developing this further in my repo, but I think this will benefit the stdlib greatly by centralizing the implementation and testing. And also the whole ecosystem will find it useful to have an already "cross platform" Libc wrapper. These tests also introduce a very easy way to figure out platform support problems.

[JoeLoser]

Hey Martin, thanks so much for pushing on the libc work! It's something the team has talked about briefly in the past (whether to include libc bindings in the stdlib or not). I'll add it again to the team's weekly design discussions this upcoming Monday and we'll circle back here based on what we discuss.

I'm sure @lsh has some thoughts on this space too 😄

[martinvuyk]

Hi @JoeLoser , still working through some kinks in MacOS here 😢 . Was planning on wrapping those up before writing a more thorough explanation of my rationale. But here it goes:

I think this is very necessary; I've already seen bindings all around the stdlib and whether we want to or not is a piece of infrastructure everyone counts on. Having it tested and actually compliant with the standard is very important especially when extending to socket work. It's a real pain-point for anyone doing low level cross platform work. While developing this I actually came across some much needed information and documentation where I actually think we'll have Windows support very soon (I already added most of the macro constants and have a way to do error logging which is the most necessary part). Also something very cool is that I've managed to make it able to be dynamically linked on the 3 operating systems as well :) (need to test windows to be sure)

PS: Eventually I'll drop another 7k ish lines 😆 with my socket design that I hope also makes those issues something of the past. Because this implementation should only deal with translating constants and function calls to be compliant with the standard, every operating system deals with socket behavior differently at the control layer.

[owenhilyard]

My main suggestion is to try to make the structure mirror the structure of POSIX, to make it easier to find things. Also, please remember /etc/protocols, there's a lot of useful things in there like PIM, mptcp (Linux only), IPv6 with no next header (nice for routers), kernel-managed TLS, and DCCP.

[msaelices]

@martinvuyk what about the sys._libc module? are they going to merge at some point?

[martinvuyk]

Hi @msaelices, yes I imagine they will merge at some point. There is also some future cleanup to switch c_int to use C.int and other such variables (if they decide to merge this PR at all).

[msaelices]

Could just is_gpu() work?

[martinvuyk]

I saw this workaround in a stdlib function, I don't know if it's some NVIDIA particular thing or if it's that the API for GPUs needs to be different 🤷‍♂️. This vprintf signature is actually non standard AFAIK since it doesn't use the va_list type, but it could also be that the underlying VariadicPack uses such a structure internally and when a Pointer traverses though it it works fine, when I tested vprintf on Linux with the same setup it did not work 🤷‍♂️.