cohttp support ?

[Jo-Blade]

I've seen in the past people tried to port cohttp package into this repository. I want to know what is the state of cross compiling projects using cohttp-lwt-unix to windows ? I want to know if it's possible and if yes, how to do ?

Thank you

[toots]

Hi!

Generally, if a project is packaged using dune, cross-compilation works pretty much out of the box unless the build needs to generate architecture-specific files using binaries ran on the local host.

Most likely, it shouldn't be a problem to add cohttp. Happy to give more details if you're looking at contributing it.

[Jo-Blade]

Hi!

Generally, if a project is packaged using dune, cross-compilation works pretty much out of the box unless the build needs to generate architecture-specific files using binaries ran on the local host.

Most likely, it shouldn't be a problem to add cohttp. Happy to give more details if you're looking at contributing it.

Thank you, I've tried packaging a bunch of packages to make it work but couldn't test in real condition because I couldn't build ssl-windows yet to test my cross compiled app with https websites (see #327 ).

I have some questions about ppx_ packages, I don't exactly understand if they need to be packaged for windows or if I can use normal ppx_ package from opam official repo ? For example, ppx_lwt for me (if I understand well) is a sort of preprocessing tool, so I don't know if I need to package it or not ?

Another question: Is there a sort of automatic process to build dune project without having to package manually all dependencies ?

[toots]

Generally speaking, when asked to build for a cross-compiled target, dune will invoke the cross-compilers unless you need to build a binary that needs to run on the host, in which case it will invoke the normal compiler.

For ppx, I believe that most of them operate using a locally run binary so you will need to have the dependencies for the host added to the opam package. Since most of them also include a library for the target architecture, you most likely also will have to have the cross-dependencies installed.

In general, you could do a trial and error process until you figure out the minimal set of dependencies. These days, I will admit that I have simply added both set of dependencies to new packages to make the process easier.

You could automatize this and systematically add both native and cross-compiled dependencies and be able to automatically create cross-compiled packages.

I still believe that it's important to do some of this manually to at least think about what you're doing. There are packages that will need a little more attention like bindings using ctypes and, for instance, generating system-specific values using locally built binaries.

On the other hand, if your package is pure OCaml code, aside from overeaching, adding both set of dependencies is pretty much guaranteed to work.

[Jo-Blade]

Now I have a new issue, I need to cross-compile ca-certs for windows (to make https working in cohttp) and it has "ptime" as a dependency. To be honest, I don't understand at all the difference between ptime and mtime…

But I couldn't succeed to build ptime. I've also discovered that mtime does not build.

mtime-windows (from this repo) fail with this error:

#=== ERROR while compiling mtime-windows.1.1.0 ================================#
# context     2.2.1 | linux/x86_64 | ocaml-base-compiler.4.14.1 | git+file:///mnt/opam-cross-windows
# path        ~/.opam/4.14.1/.opam-switch/build/mtime-windows.1.1.0
# command     /usr/sbin/env MTIME_OS=Win32 ocaml pkg/pkg.ml build --toolchain windows --pinned false --with-js_of_ocaml false
# exit-code   1
# env-file    ~/.opam/log/mtime-windows-55943-f0a6bc.env
# output-file ~/.opam/log/mtime-windows-55943-f0a6bc.out
### output ###
# Backtrace:
# [...]
# pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-toolchain' 'windows'
#      '-j' '4' '-tag' 'debug' '-build-dir' '_build' 'opam' 'pkg/META'
#      'CHANGES.md' 'LICENSE.md' 'README.md' 'src/mtime.a' 'src/mtime.cmxs'
#      'src/mtime.cmxa' 'src/mtime.cma' 'src/mtime.cmx' 'src/mtime.cmi'
#      'src/mtime.mli' 'src/mtime_top.a' 'src/mtime_top.cmxs'
#      'src/mtime_top.cmxa' 'src/mtime_top.cma' 'src/mtime_top.cmx'
#      'src/mtime_top_init.ml' 'src-os/mtime_clock.a' 'src-os/mtime_clock.cmxs'
#      'src-os/mtime_clock.cmxa' 'src-os/mtime_clock.cma'
#      'src-os/mtime_clock.cmx' 'src-os/mtime_clock.cmi'
#      'src-os/mtime_clock.mli' 'src-os/dllmtime_clock_stubs.dll'
#      'src-os/libmtime_clock_stubs.a' 'test-os/min_os.ml']: exited with 6

For ptime-windows (packaged by myself), I have this error:

#=== ERROR while compiling ptime-windows.1.1.0 ================================#
# context     2.2.1 | linux/x86_64 | ocaml-base-compiler.4.14.1 | git+file:///mnt/opam-cross-windows
# path        ~/.opam/4.14.1/.opam-switch/build/ptime-windows.1.1.0
# command     /usr/sbin/env MTIME_OS=Win32 ocaml pkg/pkg.ml build --toolchain windows --dev-pkg false
# exit-code   1
# env-file    ~/.opam/log/ptime-windows-56168-204a50.env
# output-file ~/.opam/log/ptime-windows-56168-204a50.out
### output ###
# [...]
# pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-toolchain' 'windows'
#      '-j' '4' '-tag' 'debug' '-build-dir' '_build' 'opam' 'pkg/META'
#      'CHANGES.md' 'LICENSE.md' 'README.md' 'src/ptime.a' 'src/ptime.cmxs'
#      'src/ptime.cmxa' 'src/ptime.cma' 'src/ptime.cmx' 'src/ptime.cmi'
#      'src/ptime.mli' 'src/ptime_top.a' 'src/ptime_top.cmxs'
#      'src/ptime_top.cmxa' 'src/ptime_top.cma' 'src/ptime_top.cmx'
#      'src/ptime_top_init.ml' 'src-clock/ptime_clock.a'
#      'src-clock/ptime_clock.cmxs' 'src-clock/ptime_clock.cmxa'
#      'src-clock/ptime_clock.cma' 'src-clock/ptime_clock.cmx'
#      'src-clock/ptime_clock.cmi' 'src-clock/ptime_clock.mli'
#      'src-clock/dllptime_clock_stubs.dll' 'src-clock/libptime_clock_stubs.a'
#      'src-clock/runtime.js' 'doc/index.mld' 'test/min_clock.ml']: exited with 10

I'm still investigating for finding how to solve the issue.

[toots]

The errors don't seem to say much, could you get some logs with the --verbose flag on?

[pirbo]

I've been hit by this problem. I thought I worked a workaround but I cannot find it, sorry.

So you don't redo the part of the investigation I remember, here it is:

It is a """bug"" of topkg (or more factually "any package that uses topkg and has C code has this problem").

Long story: (linux) topkg tries to be too smart and because it detects that the target is "windows", it asks (linux) ocamlbuild to build (among other files) dll[stuff]_stubs.dll (instead of dll[stuff]_stubs.so which it does when it detects that the target is linux) but (linux) ocamlbuild is not smart enough and does not know that it actually knows how to build dll files (thanks to the environment variable you've set that will make ocamlfind use cross compilers :)) so it fails (with an ugly unclear error) because of this command line argument (dll[stuff]_stubs.dll) it does not know what to do about...

Actually, I might be wrong but accordingly to my experiments, you don't need to explicitly ask for the generation of dll[stuff]_stubs.{dll,so}. It is still generated even if it is not explicitly listed in the ocamlbuild command line. The challenge is to make topkg not doing it...

[Jo-Blade]

I've been hit by this problem. I thought I worked a workaround but I cannot find it, sorry.

So you don't redo the part of the investigation I remember, here it is:

It is a """bug"" of topkg (or more factually "any package that uses topkg and has C code has this problem").

Long story: (linux) topkg tries to be too smart and because it detects that the target is "windows", it asks (linux) ocamlbuild to build (among other files) dll[stuff]_stubs.dll (instead of dll[stuff]_stubs.so which it does when it detects that the target is linux) but (linux) ocamlbuild is not smart enough and does not know that it actually knows how to build dll files (thanks to the environment variable you've set that will make ocamlfind use cross compilers :)) so it fails (with an ugly unclear error) because of this command line argument (dll[stuff]_stubs.dll) it does not know what to do about...

Actually, I might be wrong but accordingly to my experiments, you don't need to explicitly ask for the generation of dll[stuff]_stubs.{dll,so}. It is still generated even if it is not explicitly listed in the ocamlbuild command line. The challenge is to make topkg not doing it...

Thank you for your explanation ! I understand better know the problem… But I don't know anything about topkg and ocamlbuild so I don't know how to prevent it to build the .dll…

But what I don't understand too is that I probably need to build the .dll one time ? So I cannot just disable it's build, isn't it ?

In my experiments, I've also found this repository for ptime: https://github.com/routineco/ptime . It's a try of packaging ptime with dune but even if I succeed to build it using usual dune tricks, the result was not detected by ca-certs-windows… I think the c code has not been build correctly. And I don't know how dune build c code.

[toots]

You might want to consider converting the repo build system to dune. It's usually pretty easy and way faster than patching an existing one.

[Jo-Blade]

Ok I think the problem with the ptime packaged with dune I found was the missing "public_name" stanza in the dune file… Now it looks it has succeed (but can't really test it for now).

Now I have another issue with "mirage_crypto_rng" and it's "foreign_stubs":

[ERROR] The compilation of mirage-crypto-rng-windows.1.1.0 failed at "dune build -p
        mirage-crypto-rng -j 11 -x windows".

#=== ERROR while compiling mirage-crypto-rng-windows.1.1.0 ====================#
# context     2.2.1 | linux/x86_64 | ocaml-base-compiler.4.14.1 | git+file:///mnt/opam-cross-windows
# path        ~/.opam/4.14.1/.opam-switch/build/mirage-crypto-rng-windows.1.1.0
# command     ~/.opam/4.14.1/bin/dune build -p mirage-crypto-rng -j 11 -x windows
# exit-code   1
# env-file    ~/.opam/log/mirage-crypto-rng-windows-60900-8f186e.env
# output-file ~/.opam/log/mirage-crypto-rng-windows-60900-8f186e.out
### output ###
# File "rng/unix/dune", line 23, characters 9-27:
# 23 |   (names mc_getrandom_stubs))
#               ^^^^^^^^^^^^^^^^^^
# (cd _build/default.windows/rng/unix && /usr/sbin/x86_64-w64-mingw32-gcc -O2 -fno-strict-aliasing -fwrapv -mms-bitfields -O2 -fno-strict-aliasing -fwrapv -mms-bitfields -g -I /root/.opam/4.14.1/windows-sysroot/lib/ocaml -I /root/.opam/4.14.1/windows-sysroot/lib/digestif -I /root/.opam/4.14.1/windows-sysroot/lib/eqaf -I /root/.opam/4.14.1/windows-sysroot/lib/logs -I /root/.opam/4.14.1/windows-s[...]
# mc_getrandom_stubs.c:60:10: fatal error: Windows.h: No such file or directory
#    60 | #include <Windows.h>
#       |          ^~~~~~~~~~~
# compilation terminated.

I don't know how to deal with foreign_stubs… I'm browsing the web to try to find an answer.

EDIT: finally the problem seems to be "Windows.h" does not exist in linux, I have to modify to "windows.h" (it's not a problem on actual windows machine because it's not case sensitive)

[pirbo]

You can find ports of the build system of ptime and mtime (among others) at https://github.com/dune-universe/ptime and https://github.com/dune-universe/mtime .