Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support cross compiling libcartesi as a static library #156

Merged
merged 15 commits into from
Nov 21, 2023

Conversation

edubart
Copy link
Contributor

@edubart edubart commented Nov 7, 2023

Included changes:

  • introduce the TARGET_OS make parameter, to allow cross compiling to other OSes
  • make termios.h non-mandatory to compile libcartesi.a when defining NO_TERMIOS, because WASI SDK target has no support for it
  • build libcartesi.a static library
  • remove the use of std::filesystem from libcartesi.a, because WASI SDK target has no support for it
  • refactor object file lists in the makefile, to include only what it should in the static library
  • build and install libcartesi_jsonrpc.so and libcartesi_jsonrpc.a, the code was there, but it was not being compiled or installed
  • removed all uses of rpath and patchelf, now all executables and shared libraries are statically linked
  • libcartesi.so still is being built, but is not being linked by anyone, people will be able to choose to link to libcartesi.so or libcartesi.a
  • refactored make install to install and strip properly the binaries and libraries
  • some makefile cleanups
  • added possibility to compile libcartesi.a without mmap
  • isolated all OS related functions in os.cpp file with automatic detection of supported functions when targeting WASI or Windows
  • added ways to disable all OS function implementations with -DNO_TTY for example, in case the target doesn't have an OS supporting that API
  • the flags -std=c++17 -pedantic was replaced with -std=gnu++17 -Wpedantic. Because the old flags actually disabled some POSIX functions (such as mkdir and gettimeofday) in some toolchains (such as Cosmopolitan or DJGPP for DOSBox).
  • the privileged and PAGE_SIZE keywords is a define in some toolchains and gave compile errors, so I renamed them
  • added make bundle-boost, for making easy cross compiling libcartesi with other toolchains
  • allow compiling libcartesi.a in standalone without need to have GRPC or Lua installed in the system, added instructions to README, this important when only wanting to cross compile with other toolchains
  • refactored Makefile to remove some :=, to invoke some shell commands which depends on system having protobuf, grpc, lua etc only when really needed by a job
  • removed CM_API from machine-c-api-internal.h functions because those APIs should not be exposed in the public C API
  • added support for compiling libcartesi in systems without threading support with make nothreads=yes
  • fixed issues for building on MacOS with homebrew with old clang toolchain

After this PR, it is possible to compile a libcartesi.a for WASM using two different toolchains (as long you do make bundle-boost before):

With WASI SDK toolchain:

make -C src -j8 release=yes \
  TARGET_OS=Linux \
  CC=/opt/wasi-sdk/bin/clang \
  CXX=/opt/wasi-sdk/bin/clang++ \
  AR="/opt/wasi-sdk/bin/llvm-ar rcs" \
  libcartesi.a

With Emscripten toolchain:

make -C src -j8 release=yes \
  TARGET_OS=Linux \
  CC=emcc \
  CXX=em++ \
  AR="emar rcs" \
  libcartesi.a

With GCC RISC-V toolchain (for running a cartesi machine inside itself, yay):

make -C src -j8 release=yes \
  TARGET_OS=Linux \
  CC=riscv64-linux-gnu-gcc \
  CXX=riscv64-linux-gnu-g++ \
  AR="riscv64-linux-gnu-ar rcs" \
  libcartesi.a

With mingw-w64 toolchain (for Windows build):

make -C src -j8 release=yes \
    TARGET_OS=Linux \
    CC=x86_64-w64-mingw32-gcc \
    CXX=x86_64-w64-mingw32-g++ \
    AR="x86_64-w64-mingw32-ar rcs" \
    MYSOLDFLAGS="-llua.dll -static" \
    libcartesi.a cartesi.so

NOTE: TARGET_OS=Linux and cartesi.so seem suspicious, but is the correct way to cross compile to Windows build for now.

I also tested libcartesi of this PR to compile with the following platforms/toolchains:

  • FreeBSD with its GCC toolchain
  • DOS with DGJPP toolchain
  • Windows with Visual Studio clang toolchain
  • Comopolitan libc with its cosmocc toolchain (after removing exceptions)
  • MacOS 10.15 x86_64 with clang 12 + Homebrew
  • Android

@edubart edubart self-assigned this Nov 7, 2023
@edubart edubart added enhancement New feature or request refactor Restructuring code, while not changing its original functionality labels Nov 7, 2023
@edubart edubart force-pushed the feature/static-libcartesi branch 22 times, most recently from f1fdde4 to bb4158b Compare November 12, 2023 00:23
@edubart edubart changed the title feat: support compiling libcartesi as a static library feat: support cross compiling libcartesi as a static library Nov 12, 2023
@edubart edubart force-pushed the feature/static-libcartesi branch from bb4158b to 9274b3b Compare November 12, 2023 13:40
@edubart edubart force-pushed the feature/static-libcartesi branch from 01eb512 to ddc940e Compare November 14, 2023 14:52
src/Makefile Outdated Show resolved Hide resolved
@edubart edubart force-pushed the feature/static-libcartesi branch from ddc940e to 6274ec0 Compare November 16, 2023 22:01
vfusco
vfusco previously approved these changes Nov 16, 2023
src/Makefile Outdated Show resolved Hide resolved
src/Makefile Outdated Show resolved Hide resolved
src/Makefile Outdated Show resolved Hide resolved
src/Makefile Outdated Show resolved Hide resolved
src/Makefile Outdated Show resolved Hide resolved
src/Makefile Outdated Show resolved Hide resolved
src/Makefile Outdated Show resolved Hide resolved
src/Makefile Outdated Show resolved Hide resolved
Makefile Show resolved Hide resolved
src/Makefile Outdated
@@ -248,7 +248,7 @@ EMPTY:=
SPACE:=$(EMPTY) $(EMPTY)
CLANG_TIDY_HEADER_FILTER=$(PWD)/($(subst $(SPACE),|,$(LINTER_HEADERS)))

CXXFLAGS+=$(OPTFLAGS) -std=c++17 -fvisibility=hidden -fPIC -MMD $(CC_MARCH) $(INCS) $(GCFLAGS) $(UBFLAGS) $(DEFS) $(WARNS)
CXXFLAGS+=$(OPTFLAGS) -std=gnu++17 -fvisibility=hidden -fPIC -MMD $(CC_MARCH) $(INCS) $(GCFLAGS) $(UBFLAGS) $(DEFS) $(WARNS)
Copy link
Contributor

@diegonehab diegonehab Nov 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What "crazyness" required this?

Copy link
Contributor Author

@edubart edubart Nov 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some systems that I cross compiled (such as DOS), did't have all POSIX functions available when using -std=c++17, it didn't enable non standard C APIs, such as gettimeofday, mkdir, and others, because they are not in standard C. I think some systems are free to disable POSIX APIs with -std=c++17.

So having -std=gnu++17 is more friendly for cross compilation, another option could be removing -std= and let use the default one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In an ideal world, we would change this flag only on os.cpp, right?

@edubart edubart requested a review from vfusco November 21, 2023 20:57
@edubart edubart merged commit 7801dec into main Nov 21, 2023
8 checks passed
@edubart edubart deleted the feature/static-libcartesi branch November 21, 2023 20:58
@edubart edubart removed the request for review from mpolitzer November 21, 2023 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request refactor Restructuring code, while not changing its original functionality
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Support for running machines in the browser with WebAssembly
3 participants