Bytecode build failure

[code-ghalib]

I'm trying to build an empty dune project depending on owl, created using dune init proj test --libs owl, and I get this error:

$ dune build
Info: Creating file dune-project with this contents:
| (lang dune 2.7)
| (name test)
File "_none_", line 1:
Error: Error on dynamically loaded library: /home/me/.opam/4.11.1/lib/stublibs/dlleigen_cpp_stubs.so: /home/me/.opam/4.11.1/lib/stublibs/dlleigen_cpp_stubs.so: undefined symbol: _ZTVN10__cxxabiv117__class_type_infoE
Done: 37/41 (jobs: 1)

My system information is:

$ uname -srvimo
Linux 5.8.18-1-MANJARO #1 SMP PREEMPT Sun Nov 1 14:10:04 UTC 2020 x86_64 unknown GNU/Linux

Additionally, CC, CXX, LD_LIBRARY_PATH are all unset in my environment.
ldd shows this:

$ ldd /home/me/.opam/4.11.1/lib/stublibs/dlleigen_cpp_stubs.so
	linux-vdso.so.1 (0x00007ffea1dc6000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f63b0c21000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f63b0a58000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007f63b0ed2000)

Looks like libstdc++ is not there which seems surprising considering these are cpp stubs. Am I doing something wrong? Read some of the other issues that seemed related - is there a workaround short of attempting to build from source? I'm also not sure how to build from source and have it replaced inside the opam directory (not a heavy opam user). There doesn't seem to be a dockerfile for arch-linux here - is this platform not supported? Any help would be appreciated - thanks.

[code-ghalib]

Update: the problem is only with bytecode build, updating dune rules to only build native mode works.
e.g.

(test
 (name test)
 (libraries owl)
 (modes native))

Not sure if it is a platform-specific issue.

[kkirstein]

I get the same error when installing owl-zoo (on Ubuntu 20.04). It looks like -lstdc++ has been (accidentely?) removed for dlleigen_cpp_stubs (commit) and (re-)added for eigen lib itself (commit). I think it should be the other way around, as it was before.

[ryanrhymes]

#24 @mseri @kkirstein

[mseri]

foreign_library does not support c_library_flags, that has to go in the library using the stubs. Looks like we have two issues which are mutually breaking with things as they are now.

@nojb do you know if there is an alternative way for the fix from #27? I am pinging also the relative dune thread: ocaml/dune#3908

[nojb]

Did you try adding a (flags -lstdc++) field to the foreign_library stanza? If it works, I think you could remove the (c_library_flags ...) field.

[mseri]

I am doing that locally in a branch, seems to work but still testing. But then I am quite confused by the difference between the compiler/preprocessor flags and the link flags

[kkirstein]

@ryanrhymes @mseri thanks for your hints. That explains the „movement“ of linking flags. I will also test the fix on Windows when merged.

[nojb]

For info, in the future dune will do the right thing with respect to -lstdc++ it won't be necessary to add it explicitly. But the PR has not yet been merged ocaml/dune#3802. It should happen soon, though.

[mseri]

Thanks!

[mseri]

I am going to keep this open until we move to a dune with the new feature.
@kkirstein @nilsbecker can you check if the master branch of eigen works for you? If so, I am going to make a new release

[kkirstein]

Thanks for your work! Here is my feedback so far, first for Linux (Ubuntu 20.04, WSL2, OCaml 4.11.1):

opam pin add eigen https://github.com/owlbarn/eigen.git

-> eigen and owl installs without problems

Installation of owl-zoo fails:

kirk@UTCHNB121:~/Workspaces/OCaml/eigen$ opam install owl-top
The following actions will be performed:
  ∗ install owl-zoo 1.0.0 [required by owl-top]
  ∗ install owl-top 1.0.0
===== ∗ 2 =====
Do you want to continue? [Y/n] Y

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[owl-top.1.0.0] found in cache
[owl-zoo.1.0.0] found in cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[ERROR] The compilation of owl-zoo failed at "/home/kirk/.opam/4.11.1/bin/dune build -p owl-zoo -j 3".

#=== ERROR while compiling owl-zoo.1.0.0 ======================================#
# context     2.0.7 | linux/x86_64 | ocaml-base-compiler.4.11.1 | https://opam.ocaml.org#eb70a9b0
# path        ~/.opam/4.11.1/.opam-switch/build/owl-zoo.1.0.0
# command     ~/.opam/4.11.1/bin/dune build -p owl-zoo -j 3
# exit-code   1
# env-file    ~/.opam/log/owl-zoo-3261-d0f245.env
# output-file ~/.opam/log/owl-zoo-3261-d0f245.out
### output ###
# [...]
# /usr/bin/ld: /home/kirk/.opam/4.11.1/lib/eigen/libeigen_cpp_stubs.a(eigen_tensor.o):(.data.rel.ro._ZTIN5Eigen8internal27TensorLazyEvaluatorWritableINS_6DSizesIlLi4EEENS_9TensorMapINS_6TensorIdLi4ELi1ElEELi0ENS_11MakePointerEEENS_13DefaultDeviceEEE[_ZTIN5Eigen8internal27TensorLazyEvaluatorWritableINS_6DSizesIlLi4EEENS_9TensorMapINS_6TensorIdLi4ELi1ElEELi0ENS_11MakePointerEEENS_13DefaultDeviceE[...]
# /usr/bin/ld: /home/kirk/.opam/4.11.1/lib/eigen/libeigen_cpp_stubs.a(eigen_tensor.o):(.data.rel.ro._ZTIN5Eigen8internal19TensorLazyEvaluatorINS_6DSizesIlLi4EEENS_9TensorMapINS_6TensorIdLi4ELi1ElEELi0ENS_11MakePointerEEENS_13DefaultDeviceEEE[_ZTIN5Eigen8internal19TensorLazyEvaluatorINS_6DSizesIlLi4EEENS_9TensorMapINS_6TensorIdLi4ELi1ElEELi0ENS_11MakePointerEEENS_13DefaultDeviceEEE]+0x0): undefi[...]
# /usr/bin/ld: /home/kirk/.opam/4.11.1/lib/eigen/libeigen_cpp_stubs.a(eigen_tensor.o):(.data.rel.ro._ZTIN5Eigen8internal23TensorLazyBaseEvaluatorINS_6DSizesIlLi5EEEdEE[_ZTIN5Eigen8internal23TensorLazyBaseEvaluatorINS_6DSizesIlLi5EEEdEE]+0x0): undefined reference to `vtable for __cxxabiv1::__class_type_info'
# /usr/bin/ld: /home/kirk/.opam/4.11.1/lib/eigen/libeigen_cpp_stubs.a(eigen_tensor.o):(.data.rel.ro._ZTIN5Eigen8internal27TensorLazyEvaluatorReadOnlyINS_6DSizesIlLi5EEENS_9TensorMapINS_6TensorIdLi5ELi1ElEELi0ENS_11MakePointerEEENS_13DefaultDeviceEEE[_ZTIN5Eigen8internal27TensorLazyEvaluatorReadOnlyINS_6DSizesIlLi5EEENS_9TensorMapINS_6TensorIdLi5ELi1ElEELi0ENS_11MakePointerEEENS_13DefaultDeviceE[...]
# /usr/bin/ld: /home/kirk/.opam/4.11.1/lib/eigen/libeigen_cpp_stubs.a(eigen_tensor.o):(.data.rel.ro._ZTIN5Eigen8internal27TensorLazyEvaluatorWritableINS_6DSizesIlLi5EEENS_9TensorMapINS_6TensorIdLi5ELi1ElEELi0ENS_11MakePointerEEENS_13DefaultDeviceEEE[_ZTIN5Eigen8internal27TensorLazyEvaluatorWritableINS_6DSizesIlLi5EEENS_9TensorMapINS_6TensorIdLi5ELi1ElEELi0ENS_11MakePointerEEENS_13DefaultDeviceE[...]
# /usr/bin/ld: /home/kirk/.opam/4.11.1/lib/eigen/libeigen_cpp_stubs.a(eigen_tensor.o):(.data.rel.ro._ZTIN5Eigen8internal19TensorLazyEvaluatorINS_6DSizesIlLi5EEENS_9TensorMapINS_6TensorIdLi5ELi1ElEELi0ENS_11MakePointerEEENS_13DefaultDeviceEEE[_ZTIN5Eigen8internal19TensorLazyEvaluatorINS_6DSizesIlLi5EEENS_9TensorMapINS_6TensorIdLi5ELi1ElEELi0ENS_11MakePointerEEENS_13DefaultDeviceEEE]+0x0): undefi[...]
# /usr/bin/ld: /home/kirk/.opam/4.11.1/lib/eigen/libeigen_cpp_stubs.a(eigen_tensor.o):(.data.rel.ro._ZTIN5Eigen8internal27TensorLazyEvaluatorReadOnlyINS_6DSizesIlLi5EEENS_6TensorIfLi5ELi1ElEENS_13DefaultDeviceEEE[_ZTIN5Eigen8internal27TensorLazyEvaluatorReadOnlyINS_6DSizesIlLi5EEENS_6TensorIfLi5ELi1ElEENS_13DefaultDeviceEEE]+0x0): undefined reference to `vtable for __cxxabiv1::__si_class_type_info'
# /usr/bin/ld: /home/kirk/.opam/4.11.1/lib/eigen/libeigen_cpp_stubs.a(eigen_tensor.o):(.data.rel.ro._ZTIN5Eigen8internal27TensorLazyEvaluatorWritableINS_6DSizesIlLi5EEENS_6TensorIfLi5ELi1ElEENS_13DefaultDeviceEEE[_ZTIN5Eigen8internal27TensorLazyEvaluatorWritableINS_6DSizesIlLi5EEENS_6TensorIfLi5ELi1ElEENS_13DefaultDeviceEEE]+0x0): undefined reference to `vtable for __cxxabiv1::__si_class_type_info'
# /usr/bin/ld: /home/kirk/.opam/4.11.1/lib/eigen/libeigen_cpp_stubs.a(eigen_tensor.o):(.data.rel.ro._ZTIN5Eigen8internal19TensorLazyEvaluatorINS_6DSizesIlLi5EEENS_6TensorIfLi5ELi1ElEENS_13DefaultDeviceEEE[_ZTIN5Eigen8internal19TensorLazyEvaluatorINS_6DSizesIlLi5EEENS_6TensorIfLi5ELi1ElEENS_13DefaultDeviceEEE]+0x0): more undefined references to `vtable for __cxxabiv1::__si_class_type_info' follow
# collect2: error: ld returned 1 exit status
# File "caml_startup", line 1:
# Error: Error during linking (exit code 1)



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ λ build owl-zoo 1.0.0
└─
╶─ No changes have been performed

It looks like support for RTTI (runtime type info, if I am not mistaken) is missing. Are you relying on that? If not, one might disable this by adding -fno-rtti to the compiler flags.

Another question is, whether I am tho only one observing this error, e.g., do I have a special setup (not that I am aware of)?

[kkirstein]

.. and here the feedback for Windows10 (mingw toolchain in a Cygwin environment):

$ opam pin add eigen https://github.com/owlbarn/eigen.git
[eigen.0.3.0] synchronised from git+https://github.com/owlbarn/eigen.git
eigen is now pinned to git+https://github.com/owlbarn/eigen.git (version 0.3.0)

The following actions will be performed:
  ↻ recompile eigen      0.3.0*
  ↻ recompile owl        1.0.0  [uses eigen]
  ↻ recompile owl-zoo    1.0.0  [uses owl]
  ↻ recompile owl-plplot 1.0.0  [uses owl]
  ↻ recompile owl-top    1.0.0  [uses owl]
===== ↻ 5 =====
Do you want to continue? [Y/n] Y

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[owl.1.0.0] found in cache
[owl-plplot.1.0.0] found in cache
[owl-top.1.0.0] found in cache
[owl-zoo.1.0.0] found in cache
[eigen.0.3.0] synchronised from git+https://github.com/owlbarn/eigen.git

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[ERROR] The compilation of eigen failed at
        "C:\\Opt\\OCaml64\\home\\kayuwekirstein\\.opam\\4.11.1+mingw64\\bin\\dune.exe build -p eigen -j 3".

#=== ERROR while compiling eigen.0.3.0 ========================================#
# context     2.0.5 | win32/x86_64 | ocaml-variants.4.11.1+mingw64 | pinned(git+https://github.com/owlbarn/eigen.git#a80f9a00)
# path        ~/.opam/4.11.1+mingw64/.opam-switch/build/eigen.0.3.0
# command     C:\Opt\OCaml64\home\kayuwekirstein\.opam\4.11.1+mingw64\bin\dune.exe build -p eigen -j 3
# exit-code   1
# env-file    ~/.opam/log/eigen-17924-8b8a2d.env
# output-file ~/.opam/log/eigen-17924-8b8a2d.out
### output ###
#   ocamlmklib eigen_cpp/dlleigen_cpp_stubs.dll (exit 2)
# (cd _build/.sandbox/6d10c2136bd52fc96408893571ccc38e/default && C:\Opt\OCaml64\home\kayuwekirstein\.opam\4.11.1+mingw64\bin\ocamlmklib.opt.exe -g -o eigen_cpp/eigen_cpp_stubs eigen_cpp/eigen_dsmat.o eigen_cpp/eigen_spmat.o eigen_cpp/eigen_tensor.o)
# ** Fatal error: Unsupported relocation kind 0003 for __gxx_personality_seh0 in eigen_cpp/eigen_tensor.o
#     ocamlopt eigen/eigen.cmxs (exit 2)
# (cd _build/default && C:\Opt\OCaml64\home\kayuwekirstein\.opam\4.11.1+mingw64\bin\ocamlopt.opt.exe -w -40 -g -shared -linkall -I eigen -I eigen_cpp -o eigen/eigen.cmxs eigen/eigen.cmxa)
# ** Fatal error: Unsupported relocation kind 0003 for __gxx_personality_seh0 in eigen_cpp\libeigen_cpp_stubs.a(eigen_spmat.o)
# File "caml_startup", line 1:
# Error: Error during linking (exit code 2)



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ λ build eigen 0.3.0
└─
╶─ No changes have been performed
[NOTE] Pinning command successful, but your installed packages may be out of sync.

Here installation of eigen already fails. Unfortunately, I don't really have an idea what's going wrong here. Maybe it is related to this issue: link
Anyway, I would look at the Linux issue first. For me it comes down to the new foreign_library stanza in dune, which seems to handle compilation & linking flags differently.

[mseri]

Thanks for the report. Looks like these we can test here: #31
I am still unsure what is the correct way to handle this. Maybe we should wait for the dune feature (ocaml/dune#3802) to land to properly fix this and the cmxs issue at the same time

[kkirstein]

Sounds good to me. Currently, I pin eigento 0.3.0 and everything is fine for me (both on Ubuntu & Windows). Would be interested if anybody can reproduce my observations above.

[mseri]

I can reproduce the issue on a linux VM and on the GH Actions worker in the new PR

[kkirstein]

Just for completeness:

"Linux" issue: -fno-rtti does not help, I get still missing references to libstdc++ when installing owl-zoo. Looks like linking libstdc++ is somehow broken.

"Windows" issue: -fno-rtti -fno-exceptions enables installation of eigen. Installing owl-zoo fails similar to Linux issue with missing references.

[nilsbecker]

i tried on macos. i pinned eigen to a local copy of master at commit a80f9a0 . then opam upgrade completes without complaints (no owl installed in this 4.11 switch)
i then tried opam install owl, which also completed.
finally, opam install owl-top.

[ERROR] The compilation of owl-zoo failed at
        "/Users/nbecker/.opam/opam-init/hooks/sandbox.sh build dune build -p
        owl-zoo -j 4".

#=== ERROR while compiling owl-zoo.1.0.0 ======================================#
# context     2.0.7 | macos/x86_64 | ocaml-variants.4.11.1+flambda+no-flat-float-array | https://opam.ocaml.org/#7bc30c14
# path        ~/.opam/411/.opam-switch/build/owl-zoo.1.0.0
# command     ~/.opam/opam-init/hooks/sandbox.sh build dune build -p owl-zoo -j 4
# exit-code   1
# env-file    ~/.opam/log/owl-zoo-18453-520fac.env
# output-file ~/.opam/log/owl-zoo-18453-520fac.out
### output ###
# Undefined symbols for architecture x86_64:
# [...]
#       _c_eigen_dsmat_s_zeros in libeigen_cpp_stubs.a(eigen_dsmat.o)
#       _c_eigen_dsmat_s_ones in libeigen_cpp_stubs.a(eigen_dsmat.o)
#       _c_eigen_dsmat_s_eye in libeigen_cpp_stubs.a(eigen_dsmat.o)
#       _c_eigen_dsmat_s_of_bigarray in libeigen_cpp_stubs.a(eigen_dsmat.o)
#       _c_eigen_dsmat_s_print in libeigen_cpp_stubs.a(eigen_dsmat.o)
#       std::__1::basic_ostream<char, std::__1::char_traits<char> >& Eigen::operator<<<Eigen::Matrix<float, -1, -1, 1, -1, -1> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, Eigen::DenseBase<Eigen::Matrix<float, -1, -1, 1, -1, -1> > const&) in libeigen_cpp_stubs.a(eigen_dsmat.o)
#       ...
# ld: symbol(s) not found for architecture x86_64
# clang: error: linker command failed with exit code 1 (use -v to see invocation)
# File "caml_startup", line 1:
# Error: Error during linking (exit code 1)

[mseri]

@code-ghalib I have marked eigen 0.4.0 as unavailable on the opam-repository, it breaks too many workflows (surprising that it was not failing in the opam-repository CI revdeps, but I guess it is because it only tested the direct dependencies). You can still use that version by pinning dev-repo. We will release a version that fixes both linking and cmxs as soon as we find a way to have both (probably using a new version of dune)

[code-ghalib]

Thank you @mseri , just left this open just for awareness - the native build was sufficient for my needs.