Skip to content

Porting to Windows

Jump to bottom Edit New page
Quinton Miller edited this page Oct 11, 2024 · 55 revisions

Note

This page is still a work in progress. An archive of the previous instructions can be found here: Porting to Windows (old)

This page will give some help and rules for developing Crystal on Windows. Ongoing efforts will be tracked and coordinated in #5430 and the Windows Support project.

Prerequisites

The following development tools are needed if you want to rebuild Crystal entirely on Windows. Unless otherwise noted, they should all be accessible in your current user's %PATH% environment variable.

Crystal

To rebuild Crystal, you need an existing installation of Crystal. Any installer or portable package on the Releases page should work, but since Windows support is still in a flux, the most recent stable release is preferred.

Microsoft Visual Studio

Microsoft Visual C++ is currently the only toolchain supported by Crystal on Windows. The Desktop development with C++ workload should be enabled in Visual Studio Installer. Also the following individual components should be checked:

  • MSVC v14x - VS 20xx C++ x64/x86 build tools (also available individually)
  • Windows 10 / 11 SDK (also available individually)
  • C++ ATL for latest v14x build tools (x86 & x64) (required for building LLVM)

The file %ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe should be available after a successful installation. The Crystal compiler relies on its existence to be able to build executables outside the Developer Command Prompt.

GNU Make

Crystal uses GNU Make as a command runner for various development tasks.

Git for Windows

Crystal and its third-party dependencies use Git as the version control system.

CMake (optional)

CMake is needed to build LLVM and most of Crystal's third-party dependencies.

  • Website: https://cmake.org/
  • WinGet package: Kitware.CMake
  • Visual Studio Installer component: C++ CMake tools for Windows

Netwide Assembler (optional)

NASM is only required to build OpenSSL from source, and is normally not necessary since Crystal's Windows releases already include OpenSSL.

Strawberry Perl (optional)

Same as above, Strawberry Perl is only required to build OpenSSL from source. Since Strawberry Perl comes with its own MinGW tree, it is recommended to use its portableshell.bat to set up the environment on demand, rather than exposing its binary directories to %PATH%.

Inno Setup (optional)

Crystal's Windows installers are built using Inno Setup 6.

Development

All commands in this section should be run inside an x64 Native Tools Command Prompt for VS 20xx, unless otherwise noted.

Building a local development compiler natively

This is more or less the same as Linux, except the Makefile is always Makefile.win. The batch script bin/crystal.bat or PowerShell script bin/crystal.ps1 will use the standard library at the repository itself and pick up the local compiler if it exists.

git clone -c core.autocrlf=false https://github.com/crystal-lang/crystal.git
cd crystal
make -fMakefile.win crystal format release=1

Warning

The %CRYSTAL_PATH% environment variable should never be set, because Makefile.win populates the freshly built compiler with the same defaults as other platforms. If you have defined it system-wide in order to make previous compiler versions work, now is the time to unset it.

Warning

When building a 1.12.0-dev or higher compiler with crystal-lang/crystal#14292 applied, using a base compiler without it, either FLAGS=-Dpreview_dll or static=1 must be specified, otherwise the build will be confused about whether it is linking dynamically or statically.

Building third-party dependencies

These steps are also taken from .github/workflows/win.yml. Start a PowerShell session within the Developer Command Prompt, then run the following at the repository's root directory:

.\etc\win-ci\build-gc.ps1 -BuildTree deps\gc -Version 8.2.6 -AtomicOpsVersion 7.8.2
.\etc\win-ci\build-pcre.ps1 -BuildTree deps\pcre -Version 8.45
.\etc\win-ci\build-pcre2.ps1 -BuildTree deps\pcre2 -Version 10.43
.\etc\win-ci\build-iconv.ps1 -BuildTree deps\iconv
.\etc\win-ci\build-ffi.ps1 -BuildTree deps\ffi -Version 3.3
.\etc\win-ci\build-z.ps1 -BuildTree deps\z -Version 1.3.1
.\etc\win-ci\build-mpir.ps1 -BuildTree deps\mpir
.\etc\win-ci\build-yaml.ps1 -BuildTree deps\yaml -Version 0.2.5
.\etc\win-ci\build-xml2.ps1 -BuildTree deps\xml2 -Version 2.12.5
.\etc\win-ci\build-openssl.ps1 -BuildTree deps\openssl -Version 3.1.0
.\etc\win-ci\build-llvm.ps1 -BuildTree deps\llvm -Version 18.1.1 -Dynamic

Dynamic libraries are produced if -Dynamic is also supplied, otherwise static libraries are produced. The above will place the static or import libraries under libs/, and DLLs under dlls/. Since Windows doesn't have a pkg-config equivalent to detect library version at compilation time, some additional files are also part of the builds:

  • libs/openssl_VERSION contains the OpenSSL version number determined at build time.
  • libs/llvm_VERSION contains the LLVM version number, targets built, and required Win32 libraries, all determined at build time.

Note

Building LLVM statically isn't supported yet (but see Static linking below).

Note

Currently, MPIR relies on a third-party GitHub repository.

Building an installer

To build the Windows installer, all the necessary files should be prepared under etc/win-ci/portable with the same structure as a portable package:

  • docs/*: standard library documentation
  • examples/*: sample programs
  • lib/*: static or import libraries, plus version files
  • src/*: standard library
  • crystal.exe: the compiler
  • crystal.pdb: the compiler's debug symbols
  • LICENSE.txt: compiler license
  • README.md: compiler readme
  • shards.exe: the dependency manager
  • shards.pdb: the dependency manager's debug symbols
  • *.dll: dynamic libraries

Then run iscc.exe crystal.iss inside etc/win-ci/. Refer to the GitHub workflow jobs for details.

Static linking

All third-party dependencies can be statically linked by passing static=1 to make. However, because LLVM's static libraries are extremely huge, they are not distributed with the Windows packages and must be built from source. The following build instructions, which should work on the stock PowerShell console that comes with Windows, are taken from .github/workflows/win.yml:

git clone --config core.autocrlf=false -b llvmorg-x.y.z --depth 1 https://github.com/llvm/llvm-project.git
mkdir llvm-build
cd llvm-build
cmake ..\llvm-project\llvm -Thost=x64 -DLLVM_TARGETS_TO_BUILD="X86;AArch64" -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DBUILD_SHARED_LIBS=OFF -DCMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH=OFF -DLLVM_INCLUDE_BENCHMARKS=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_ENABLE_ZSTD=OFF
cmake --build . --config Release -j8
cmake "-DCMAKE_INSTALL_PREFIX=$(pwd)\..\llvm" "-DCMAKE_INSTALL_CONFIG_NAME=Release" -P cmake_install.cmake

where x.y.z is the LLVM version number, e.g. 17.0.6. A successful build should produce a file called ...\llvm\bin\llvm-config.exe. The %LLVM_CONFIG% environment variable should point to this executable, so that Crystal knows which version of LLVM it is building against.

Note

%LLVM_CONFIG% is entirely ignored during dynamic linking. There are no plans to support the find-llvm-config script with the MSVC toolchain.

Updating the LLVM DLL

scripts/generate_llvm_version_info.cr accepts an LLVM DLL path or locates it from %PATH%, and then prints output that can be used as libs/llvm_VERSION. If LLVM is installed using e.g. their volunteer-built Windows installers, this allows replacing the LLVM-C.dll and libs/llvm_VERSION files in a Crystal package without obtaining the LLVM source code.

Cross-compiling Crystal

It is still possible to cross-compile Crystal for Windows from a non-Windows system. Note that the Crystal repository has no provisions for cross-compiling Crystal's third-party dependencies, so you still need an existing Crystal package to obtain those libraries. First, build crystal.obj from the non-Windows host using:

CRYSTAL_LIBRARY_PATH=... make target=x86_64-windows-msvc FLAGS=-Dpreview_dll

CRYSTAL_LIBRARY_PATH must include the same lib directory as a Windows Crystal package, since the standard library currently uses the version file there to determine the LLVM version at cross-compilation time. The resulting linker command is not directly usable; the /LIBPATHs should be updated, and the static libraries should be replaced with their import library equivalents. Now link crystal.obj on Windows:

cl.exe /nologo crystal.obj /Fecrystal /link /DEBUG:FULL /PDBALTPATH:%_PDB% /INCREMENTAL:NO /STACK:0x800000 \
  /LIBPATH:... psapi.lib shell32.lib ole32.lib uuid.lib advapi32.lib \
  llvm-dynamic.lib pcre2-8-dynamic.lib gc-dynamic.lib \
  /ENTRY:wmainCRTStartup msvcrt.lib iconv.lib mswsock.lib advapi32.lib vcruntime.lib shell32.lib ole32.lib WS2_32.lib kernel32.lib Kernel32.lib legacy_stdio_definitions.lib DbgHelp.lib ucrt.lib

Alternatively, if the host platform is WSL, it is also possible to invoke lld-link on WSL itself while using the Windows libraries directly: (the syntax is slightly different as lld-link doesn't support compiler flags)

lld-link /nologo crystal.obj /out:crystal.exe /DEBUG:FULL /PDBALTPATH:%_PDB% /INCREMENTAL:NO /STACK:0x800000 \
  /LIBPATH:... psapi.lib shell32.lib ole32.lib uuid.lib advapi32.lib \
  llvm-dynamic.lib pcre2-8-dynamic.lib gc-dynamic.lib \
  /ENTRY:wmainCRTStartup msvcrt.lib iconv.lib mswsock.lib advapi32.lib vcruntime.lib shell32.lib ole32.lib WS2_32.lib kernel32.lib Kernel32.lib legacy_stdio_definitions.lib DbgHelp.lib ucrt.lib \
  "/LIBPATH:/mnt/c/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/lib/x64" \
  "/LIBPATH:/mnt/c/Program Files (x86)/Windows Kits/10/lib/10.0.22621.0/ucrt/x64" \
  "/LIBPATH:/mnt/c/Program Files (x86)/Windows Kits/10/lib/10.0.22621.0/um/x64"

Guidelines

  • Conditionally compiled code should use flag?(:win32) for code using the Win32 API, flag?(:windows) for any code on the Windows platform (including MinGW-w64 in the future), and flag?(:msvc) for code directly related to the MSVC tools (or the Microsoft C++ ABI). When in doubt, flag?(:win32) will usually do.
  • Bindings for Win32 and C runtime APIs should be placed under src/lib_c/x86_64-windows-msvc/c. They should use LibC and their filenames should follow the original C headers where they are defined. (The bindings make no distinction between the shared, ucrt, and um subdirectories in the Windows SDK.)
  • All bindings should follow their original names as closely as possible. In particular, fun names should not be snakecased.
  • Specs should be marked with pending_win32 if they are supposed to work on Windows, but do not yet for reasons. Specs that aren't supposed to work at all should be disabled with {% unless flag?(:win32) %} instead. (There are only very few of these remaining in the whole repository.)
Add a custom footer
Add a custom sidebar
Clone this wiki locally