does dune enforce complete cmxs files? #3908
Comments
|
When you have a shared library (under Unix, a Moreover, linking a static library into a shared object is somewhat tricky and system- and platform-dependent, so it is not a very common operation. Having said that, you should be able to dynlink |
|
This is not a problem that requires Simply said trying to use that The missing symbol can be found in So you need to either:
@nilsbecker confirmed 2. is not being done and on macOS here an
Note that in the particular case of stubs, it's not really a library to be used by many different entities, usually the stubs are "owned" by a corresponding OCaml library they are not meant to be used separately so I think I'd rather put the stubs in the That being said maybe @bobot who certainly has more experience about these things may know what the best course of action between 1. and 2. above is. |
|
Hi Daniel,
I don't think 2 works: if you try that command-line, the static library will just be ignored. One issue is that there is no portable way to perform that operation (link a static library into a shared object). Do you know how to do 1.? |
I think this should work (it doesn't) since that is the way the More interestingly however, looking at the following invocation. It is in fact already mentioned on the cli via I'm staring to wonder if this is not an macos issue. Could someone
Not sure but I think you should simply mention the libraries via But let's not do that, these things in |
I'd be really curious about that now. Reading a bit Adding that option: the symbols seems to be all in (by the simple virtue of the If this exactly what happens on linux by default then maybe we should work something upstream. |
|
ok, i was able to try this on a linux system, i installed a fresh 4.11.1 switch, did gives i can try with other linker options if someone tells me which ones! |
|
@nilsbecker I'm sorry I don't have the time to look into this myself on a linux box at the moment. But I'd like to know the C linker invocation and what happens on linux with the resulting symbols in the cmxs (use |
|
ok, i went into the lib install directory of the opam switch on the linux box. somehow '_stubs' is auto-appended. i tried without the suffix '_stubs' but that cmxa file does not exist. i also tried your second invocation. this produced the same error |
|
this is the complete 'lib' subdirectory: |
No I likely gave you the wrong invocation you need to include the dir (cf. what So try something like: If that doesn't work force the library directory with |
|
ok that's better. i changed the subdirectory name: next i ran i then ran and is this what you were looking for? |
|
Well it seems the behaviour of the linker is consistent with macOS's one but I don't know how to convince it the same way to force the link (I thought maybe that was businness for The problem may be (?) that nothing mentions the stubs as |
This is exactly right, thanks for bringing it up -- I had forgotten about it. Indeed if the externals are not mentioned they will typically not be linked in. For the case at hand, the solution is easy. As mentioned by Daniel, the problem is that you are creating an empty OCaml library ( diff --git a/eigen/dune b/eigen/dune
index 8d7a016..f8e1e56 100644
--- a/eigen/dune
+++ b/eigen/dune
@@ -31,13 +31,14 @@ let () = Printf.ksprintf Jbuild_plugin.V1.send {|
(library
(name eigen)
(public_name eigen)
- (libraries ctypes eigen.cpp)
+ (libraries ctypes)
(modules :standard ffi_eigen_generated)
+ (foreign_archives ../eigen_cpp/eigen_cpp_stubs)
(foreign_stubs
(language c)
(names eigen_utils_stubs ffi_eigen_generated_stub)
- (flags :standard %s -I../eigen_cpp/lib)
+ (include_dirs ../eigen_cpp/lib)
+ (flags :standard %s)
)
- (c_library_flags :standard)
)
|} eigen_flags
diff --git a/eigen_cpp/dune b/eigen_cpp/dune
index 17cad9e..6ac18a1 100644
--- a/eigen_cpp/dune
+++ b/eigen_cpp/dune
@@ -16,26 +16,21 @@ let () = Printf.ksprintf Jbuild_plugin.V1.send {|
(include_subdirs unqualified)
-(library
- (name eigen_cpp_stubs)
- (public_name eigen.cpp)
- (libraries ctypes)
- (foreign_stubs
- (language cxx)
- (names eigen_tensor eigen_dsmat eigen_spmat)
- (include_dirs lib lib/unsupported)
- (flags
- :standard
- -fPIC
- -ansi
- -pedantic
- -O3
- -std=c++11
- %s
- %s
- )
+(foreign_library
+ (archive_name eigen_cpp_stubs)
+ (language cxx)
+ (names eigen_tensor eigen_dsmat eigen_spmat)
+ (include_dirs lib lib/unsupported)
+ (flags
+ :standard
+ -fPIC
+ -ansi
+ -pedantic
+ -O3
+ -std=c++11
+ %s
+ %s
)
- (c_library_flags :standard -lstdc++)
)
|} devflags optflags |
|
thanks @nojb i was trying to apply your patch by copy and pasting it and then from a local clone of the eigen repo, doing |
Should work either way. Check out https://github.com/nojb/eigen |
|
ok, checking out @nojb's clone worked. i then removed the stubs line from @dbuenzli load.ml test program: and ran so this seems to be able to load eigen.cmxs. this is on a macos machine without the native toplevel installed, so i did not test that. |
|
i retried the same on the linux machine. cloned into with the same shortened load.ml file. ?!?!? |
|
i should add, the macos machine has ocaml 4.10, the linux machine 4.11. |
|
i installed a fresh 4.11 switch on the macos machine and tried the same. this also succeeds. |
|
On 10 November 2020 at 17:08:00, nilsbecker ***@***.***) wrote:
error loading shared library: Dynlink.Error (Dynlink.Cannot_open_dll "Failure(\"/home/beckern/.opam/411/lib/eigen/eigen.cmxs:
undefined symbol: _ZTVN10__cxxabiv117__class_type_infoE\")")
```
This [seems][seems] to be a `libstdc++` thing. Is maybe the c++ lib install structure different on linux and macos ?
Try to ldd (on macos use `otool -L`) the cmxs to see if the libstdc++ library is linked also check the cmxa with ocamlobjinfo that it has the -lstdc++.
[seems]: https://jonnoftw.github.io/2017/01/27/linux-linker-error-_ztvn10__cxxabiv117__class_type_infoe
Daniel
|
|
i'll try. first another data point: on macos everything seems to be working. on another macos machine, this one with a switch that has ocamlnat, it was able to load the patched eigen with Omod. nice! (i guess this was expected) |
|
and even |
|
My patch may be responsible, try adding back the |
|
on the linux machine: so no libstdc++ ? |
|
aha! the patched patch now works on linux. and: so this last patch seems to do it. |
|
i'll report this back to the owl folks. but a more general question would be if problems of this kind could be somehow spotted directly when libraries are published? could there be some automatic testing that dynlink works? or some way to help library developers test for that? or can dune automatically do everything right (TM) ? |
|
incidentally, i don't know how to set up ocaml 4.11 to install ocamlnat. it seems the makefile organisation in the distribution has changed. does anybody here know? |
|
@nilsbecker congratulations, you are now an OCaml linker expert.
Honestly I think the OCaml compiler is not helping here. I think that in the It seems on macOS we can do this, I'm unsure what can done for I'll let @nojb comment whether this is worth opening an issue upstream. |
Sure! Having a summary of what the problem is would be useful, since it is something that comes up periodically. |
|
Thanks! On that note, I will close this issue, re-open if needed. |
|
Looks like the fix for the EDIT: seems to work if I move |
|
Can we reopen this? Moving the flags has broken other things. Looks like the current behaviour of dune with C++ foreign libraries is very fragile. |
cc @snowleopard |
|
Thanks for tagging me @nojb! This is a long thread, which seems to have explored a couple of dead-ends. Since you've been involved in it from the start, could you summarise what was discovered? Is there a consensus about what needs to be done to make Dune's support fo C++ foreign stubs less fragile? |
Sure! This is my understanding of the current situation. @nilsbecker / @mseri will correct me if I'm wrong. The challenge is to build the library Currently, there is a partial solution that puts the
So a good first step would be to figure out if there is a way to make everything work at once. |
|
In fact, it is a bit worse than that: adding So the old mechanism of having a dummy The fix for |
This is a follow-up to #3889 .
After some iterations it turned out that the file
eigen_cpp_stub.cmxsin the Eigen library does not contain the necessary C stubs. This may be due to automatic dune build rules being insufficient for generating complete .cmxs files. if so, that could be considered a bug in dune.Expected Behavior
Eigen can be loaded in the native ocaml toplevel via Omod and can be dynamically linked to native code via Dynlink.
Actual Behavior
Loading via Omod fails, as reported in #3889 (comment)
Reproduction
#use "omod.nattop";; Omod.load "Eigen";;alternatively, the same issue should also appear when attempting to link eigen in native code via Dynlink. that may be easier to reproduce since it does not require installing ocamlnat. however, i have not tried.
Specifications
see also these issue comments:
owlbarn/eigen#27 (comment)
dbuenzli/omod#12 (comment)
The text was updated successfully, but these errors were encountered: