Add support directly to wasm4

[tjpalmer]

Here's the issue there that I made for this. This is just to point back that way.

Also, do I need this separately for building? https://github.com/dotnet/runtimelab/tree/feature/NativeAOT-LLVM

[aduros]

Thanks for digging into this! Very exciting 😄

Instead of building llvm, would it be possible to use wasm-ld or lld from the WASI SDK?

[yowl]

@aduros Yes, that works. E.g. I used Windows Subsystem for Linux (MinGW might also work), and

./wasm-ld -o /mnt/e/GitHub/w4/hello-wasm.wasm /mnt/e/GitHub/w4/hello.bc /mnt/e/GitHub/w4/MiniRuntime.bc /mnt/e/GitHub/w4/helloclrjit.bc -mllvm -combiner-global-alias-analysis=false -mllvm -disable-lsr --import-undefined --strip-debug --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=fflush  --export-table -z stack-size=1024 --import-memory --initial-memory=65536 --max-memory=65536 --global-base=6560 --export=update --no-entry

Probably some of these options can be removed, the __start/__stop_em ones in particular. They look like leftovers from me copying the emscripten commands which we don't need as we're not using emscripten.

@tjpalmer Yes, you need to build NativeAOT-LLVM unfortunately. In particular as of today there are 2 barriers to entry here.

• The NativeAOT-LLVM package building is broken so you need to build from source. That issue can be tracked at [NativeAOT-LLVM] Can we produce package artifacts dotnet/runtimelab#1792
• The https://github.com/dotnet/runtimelab/tree/feature/NativeAOT-LLVM branch is missing the feature to build libraries and has a bug in calling back to wasm4 whereby it leaks stack and crashes quickly. Both these issues are fixed in NativeAOT-LLVM: Support nativelib for Wasm  dotnet/runtimelab#1791

[tjpalmer]

Thanks for the info! Out of curiosity, do string literals work in no runtime mode? If not, do char arrays work?

[tjpalmer]

Meanwhile, I'm thinking a dockerfile might be best for figuring out how to make this work and for helping others run it.

[yowl]

The problem with string literals is that in C# they are unicode (UTF16), so if you want to pass that to text for example, then you've got work to do. You could I suppose use fixed to get a byte array and then copy every other char to some new byte[], which you'd get away with for low unicode chars where the unicode is the same as the ASCII. A text based adventure game might benefit from that. If you've just got a handful of literals, maybe not.

[yowl]

Sort of like this

public struct TwoByteStr
{
    public byte first;
    public byte second;
}

    private static unsafe void PrintString(string s)
    {
        int length = s.Length;
        fixed (char* curChar = s)
        {
            for (int i = 0; i < length; i++)
            {
                TwoByteStr curCharStr = new TwoByteStr();
                curCharStr.first = (byte)(*(curChar + i));
                printf((byte*)&curCharStr, null);
            }
        }
    }

But that is doing one at a time.

[tjpalmer]

Glad to hear it might work at all, though. Thanks for the info. (Edit: My main concern was that string literals might not work without string objects and that those might not work without runtime support. But apparently they can. I still haven't gotten that far in setting things up, so I was just asking in advance.)

[tjpalmer]

I started trying to work a dockerfile for this.

Looks like I forgot to push my latest changes, though, and I'm on a different system at the moment. When I run build.sh in the runtimelab, it grabs dotnet all over again, so I commented out the first round. But the build failed after some while, and I didn't have time to look into it. I might should install clang first, but I'm not sure if that was the issue.

I'll try looking into it more later.

[yowl]

Ah yes, the Linux build has rotted since the compiler moved repos . It used to work so shouldn't be hard to get it going . I'll have a look later.

…

-- Scott

________________________________ From: Tom ***@***.***> Sent: Thursday, December 23, 2021 7:41:58 AM To: yowl/csharp-wasm4 ***@***.***> Cc: Scott Waye ***@***.***>; Comment ***@***.***> Subject: Re: [yowl/csharp-wasm4] Add support directly to wasm4 (Issue #1) I started trying to work a dockerfile for this<https://github.com/tjpalmer/wasm4-docker/blob/dotnet/cs/Containerfile>. Looks like I forgot to push my latest changes, though, and I'm on a different system at the moment. When I run build.sh in the runtimelab, it grabs dotnet all over again, so I commented out the first round. But the build failed after some while, and I didn't have time to look into it. I might should install clang first, but I'm not sure if that was the issue. I'll try looking into it more later. — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAUYCKJQLBOB5DSYHX5PRVLUSMKJNANCNFSM5KSSIWJA>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

[tjpalmer]

In case it matters, this is where it dies:

/work/runtimelab/eng/python.targets(21,5): error MSB3073: The command "..." exited with code 127. [/work/runtimelab/src/coreclr/System.Private.CoreLib/System.Private.CoreLib.csproj]

[tjpalmer]

And I don't have python on it yet. Maybe should do that first.

[tjpalmer]

After adding clang, cmake, make, and python3, it now works until this happens:

    All projects are up-to-date for restore.
  Invoking "/work/runtimelab/eng/native/gen-buildsys.sh" "/work/runtimelab/src/coreclr" "/work/runtimelab/artifacts/obj/coreclr/Linux.x64.Debug" x64 clang "" "" Debug ""  -DCLR_CMAKE_PGO_INSTRUMENT=0 -DCLR_CMAKE_OPTDATA_PATH=/root/.nuget/packages/optimization.linux-x64.pgo.coreclr/1.0.0-prerelease.21452.5 -DCLR_CMAKE_PGO_OPTIMIZE=1 -DFEATURE_DISTRO_AGNOSTIC_SSL=1 
  CMake command line is unchanged. Reusing previous cache instead of regenerating.
  /work/runtimelab/artifacts/obj/coreclr/Linux.x64.Debug /work/runtimelab/src/coreclr
  Executing make install -j 8
  make: *** No rule to make target 'install'.  Stop.

[yowl]

Ok, sounds good. There are some docker images for building the runtime with all the prerequisites but looks like you've got past that, https://github.com/dotnet/runtimelab/blob/feature/NativeAOT-LLVM/docs/workflow/requirements/linux-requirements.md

Failing in gen-buildsys.sh sounds about right. I'm away for a few days and my SSD destroyed the OS, so I've been a bit behind. NativeAOT-LLVM used to be part of CoreRT where the Linux build was working so there might be some clues in https://github.com/dotnet/corert/blob/master/src/Native/gen-buildsys-clang.sh if you fancy it . Otherwise I'll look when I get back,

[aduros]

The problem with string literals is that in C# they are unicode (UTF16), so if you want to pass that to text for example, then you've got work to do.

We have wide variants of all functions that take strings, eg: textUtf16 that probably can be used here. See the note at https://wasm4.org/docs/reference/functions#text-str-x-y

[yowl]

@aduros Thanks, I'll get that in the sample
@tjpalmer I made some progress but hit a problem with errno.h building part of the compiler. I've created dotnet/runtimelab#1797 for that as I'm not sure what direction it should be going.

[tjpalmer]

I made some progress but hit a problem with errno.h building part of the compiler. I've created dotnet/runtimelab#1797 for that as I'm not sure what direction it should be going.

Thanks much for working on this! Do you also have a dockerfile by chance?

As an aside, I'm likely to switch base distro from Alpine to Ubuntu or something because wasi-sdk binaries seem to have trouble on Alpine.

[yowl]

The Windows packages are now being built, not much help for the Linux side, and we still need the lib PR merged for it to be useful for wasm4.

https://dev.azure.com/dnceng/public/_packaging?_a=feed&feed=dotnet-experimental

I'm using Ubuntu under WSL to try to get it building, but I've never created a docker file. Not sure whats involved there, would need to read up on it.

[tjpalmer]

I'm using podman (rather than docker) on linux. I'm not sure what's required for setup on windows. My command on linux is to go into the dir with the containerfile and run:

podman build -t wasm4-dotnet .

[tjpalmer]

That's building the image, not using it, but if that doesn't end in error, that's still progress.

[yowl]

The windows packages are now available at https://dev.azure.com/dnceng/public/_packaging?_a=feed&feed=dotnet-experimental These contain the compiler that can build the library as required by wasm4.

Unfortunately, although I spent some time trying to get the Linux build going, I've not succeeded and I'm unlikely to make much progress on it before the game jam.

[tjpalmer]

Thanks much for working on this. I'm unlikely to work at it before the jam either. Maybe next time. And it'd be nice if they just get this feature set into main .net at some point.