GMP: New version 6.3.0 #7872

eschnett · 2023-12-29T20:07:33Z

No description provided.

G/GMP/[email protected]/build_tarballs.jl

eschnett · 2023-12-30T17:03:09Z

@maleadt @ViralBShah There is a patch for GMP 6.2.1 that handles "overflow" (out of memory). The respective code changed in GMP 6.3.0. The older patches don't apply, and I don't know whether the patch should be updated or whether it is now superfluous.

eschnett · 2023-12-30T17:35:51Z

Since Julia Base calls __gmp_set_alloc_overflow_function I decided to undo the changes to GMP that now handle mpz overflow via an error code in favour of the mechanism introduced in our patches. The main reasons being (a) it will work for Julia right away, and (b) our patches handle more cases.

maleadt · 2024-01-01T13:49:56Z

I did add tests for the overflow cases I encountered to JuliaLang/julia#51243, so as long as Julia tests pass it should be OK to go with upstream.

giordano · 2024-01-02T16:21:08Z

Msan build is failing with

[16:11:08] configure:11251: checking C++ compiler c++  -g -O2
[16:11:08] Test compile:
[16:11:08] configure:11265: c++  -g -O2 conftest.cc >&5
[16:11:08] /opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld: warning: libc.musl-x86_64.so.1, needed by /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0/../../../../x86_64-linux-musl/lib/../lib64/libstdc++.so, not found (try using -rpath or -rpath-link)
[16:11:08] /opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld: a.out: hidden symbol `fstat' in /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib64/libc_nonshared.a(fstat.oS) is referenced by DSO
[16:11:08] /opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld: final link failed: Bad value
[16:11:08] clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
[16:11:08] configure:11268: $? = 1
[16:11:08] failed program was:
[16:11:08] 
[16:11:08] int main (void) { return 0; }
[16:11:08] configure:11380: result: no
[16:11:08] configure:11396: error: C++ compiler not available, see config.log for details

eschnett · 2024-01-14T16:57:39Z

Compiling a simple C++ program

int main (void) { return 0; }

works. Linking it fails:

sandbox:${WORKSPACE}/srcdir/gmp-6.3.0 # c++  --verbose -O2 -pedantic -fomit-frame-pointer -m64 conftest.o
clang version 13.0.1 (/home/mose/.julia/dev/BinaryBuilderBase/deps/downloads/clones/llvm-project.git-1df819a03ecf6890e3787b27bfd4f160aeeeeacd50a98d003be8b0893f11a9be 75e33f71c2dae584b13a7d1186ae0a038ba98838)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/x86_64-linux-musl/bin
Found candidate GCC installation: /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0
Selected GCC installation: /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/opt/bin/x86_64-linux-gnu-libgfortran3-cxx11-sanitize+memory/ld.x86_64-linux-gnu" --sysroot=/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root -z now -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib/../lib64/crt1.o /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib/../lib64/crti.o /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0/crtbegin.o -L/opt/x86_64-linux-gnu/lib/gcc/opt/x86_64-linux-gnu/lib/gcc -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib -L/opt/x86_64-linux-gnu/lib/gcc/x86_64-linux-gnu/6.1.0 -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/lib -L/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0 -L/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0/../../../../x86_64-linux-musl/lib/../lib64 -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/lib/../lib64 -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib/../lib64 -L/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0/../../../../x86_64-linux-musl/lib -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/lib -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib --whole-archive /opt/x86_64-linux-musl/lib/clang/13.0.1/lib/linux/libclang_rt.msan-x86_64.a --no-whole-archive --whole-archive /opt/x86_64-linux-musl/lib/clang/13.0.1/lib/linux/libclang_rt.msan_cxx-x86_64.a --no-whole-archive --export-dynamic conftest.o -lstdc++ -lm --no-as-needed -lpthread -lrt -lm -ldl -lgcc_s -lgcc -lc -lgcc_s -lgcc /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0/crtend.o /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib/../lib64/crtn.o
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld: cannot find /opt/x86_64-linux-musl/lib/clang/13.0.1/lib/linux/libclang_rt.msan-x86_64.a: No such file or directory
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld: cannot find /opt/x86_64-linux-musl/lib/clang/13.0.1/lib/linux/libclang_rt.msan_cxx-x86_64.a: No such file or directory
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)

It is weird that the linker is trying to pull in files (crtbegin.o, libclang_rt.msan-x86_64.a etc.) from the directory /opt/x86_64-linux-musl. I would expect it to use the files to be in /opt/x86_64-linux-gnu instead.

giordano · 2024-01-14T19:43:48Z

That's because you probably didn't get the MSAN runtime from LLVMCompilerRT, we do in our build: https://buildkite.com/julialang/yggdrasil/builds/7762#018d0979-ae62-4dba-89bb-58827f2d5dfe/609-3173.

The problem is

[16:11:08] configure:11265: c++  -g -O2 conftest.cc >&5
[16:11:08] /opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld: warning: libc.musl-x86_64.so.1, needed by /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0/../../../../x86_64-linux-musl/lib/../lib64/libstdc++.so, not found (try using -rpath or -rpath-link)
[16:11:08] /opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld: a.out: hidden symbol `fstat' in /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib64/libc_nonshared.a(fstat.oS) is referenced by DSO
[16:11:08] /opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld: final link failed: Bad value
[16:11:08] clang-13: error: linker command failed with exit code 1 (use -v to see invocation)

somehow the linker is trying to use musl libraries.

eschnett · 2024-01-15T21:59:10Z

Is this related?

giordano · 2024-01-16T17:24:39Z

Is this related?

I missed that, but I don't think it's related, should have just used llvm 13 until we can build compiler-rt for llvm 16 (#7877)

giordano · 2024-01-27T12:25:49Z

sandbox:${WORKSPACE}/srcdir/gmp-6.3.0 # echo 'int main (void) { return 0; }' | SUPER_VERBOSE=1 c++ -x c++ - -g -O2
ccache /opt/x86_64-linux-musl/bin/clang++ -target x86_64-linux-gnu --sysroot=/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root -D_GLIBCXX_USE_CXX11_ABI=1 -stdlib=libstdc++ -march=x86-64 -mtune=generic -fsanitize=memory -isystem /opt/x86_64-linux-gnu/x86_64-linux-gnu/include/c++/6.1.0 -isystem /opt/x86_64-linux-gnu/x86_64-linux-gnu/include/c++/6.1.0/x86_64-linux-gnu -isystem /opt/x86_64-linux-gnu/x86_64-linux-gnu/include/c++/6.1.0/backward -isystem /opt/x86_64-linux-gnu/x86_64-linux-gnu/include -isystem /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/include -x c++ - -g -O2 -rtlib=libgcc -L/opt/x86_64-linux-gnu/lib/gcc/opt/x86_64-linux-gnu/lib/gcc -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib -L/opt/x86_64-linux-gnu/lib/gcc/x86_64-linux-gnu/6.1.0 -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/lib -fuse-ld=x86_64-linux-gnu -fsanitize=memory
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld --sysroot=/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root -z now -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib/../lib64/crt1.o /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib/../lib64/crti.o /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0/crtbegin.o -L/opt/x86_64-linux-gnu/lib/gcc/opt/x86_64-linux-gnu/lib/gcc -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/lib -L/opt/x86_64-linux-gnu/lib/gcc/x86_64-linux-gnu/6.1.0 -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/lib -L/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0 -L/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0/../../../../x86_64-linux-musl/lib/../lib64 -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/lib/../lib64 -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib/../lib64 -L/opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0/../../../../x86_64-linux-musl/lib -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/lib -L/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib --whole-archive /opt/x86_64-linux-musl/lib/clang/13.0.1/lib/linux/libclang_rt.msan-x86_64.a --no-whole-archive --dynamic-list=/opt/x86_64-linux-musl/lib/clang/13.0.1/lib/linux/libclang_rt.msan-x86_64.a.syms --whole-archive /opt/x86_64-linux-musl/lib/clang/13.0.1/lib/linux/libclang_rt.msan_cxx-x86_64.a --no-whole-archive --dynamic-list=/opt/x86_64-linux-musl/lib/clang/13.0.1/lib/linux/libclang_rt.msan_cxx-x86_64.a.syms /tmp/--fa2059.o -lstdc++ -lm --no-as-needed -lpthread -lrt -lm -ldl -lgcc_s -lgcc -lc -lgcc_s -lgcc /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0/crtend.o /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib/../lib64/crtn.o
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld: warning: libc.musl-x86_64.so.1, needed by /opt/x86_64-linux-musl/bin/../lib/gcc/x86_64-linux-musl/6.1.0/../../../../x86_64-linux-musl/lib/../lib64/libstdc++.so, not found (try using -rpath or -rpath-link)
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld: a.out: hidden symbol `fstat' in /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/lib64/libc_nonshared.a(fstat.oS) is referenced by DSO
/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu-ld: final link failed: Bad value
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)

If I understand it correctly, the linker invocation looks extremely wrong, it's mixing up glibc and musl directories and object files.

giordano · 2024-01-27T12:44:08Z

Uhm, this works on JuliaPackaging/BinaryBuilderBase.jl@fec32a3, but not JuliaPackaging/BinaryBuilderBase.jl@b52b7a1. It's bisection time.

giordano · 2024-01-27T13:20:33Z

@gbaraldi this was broken around JuliaPackaging/BinaryBuilderBase.jl@65d6001 (JuliaPackaging/BinaryBuilderBase.jl#324), but honestly I can't quite tell why. Note that on that specific commit the C compiler is also broken, this was then fixed by JuliaPackaging/BinaryBuilderBase.jl@e76628b (JuliaPackaging/BinaryBuilderBase.jl#332), here only the C++ compiler is broken, so it's a bit complicated to figure out what exactly caused the breakage.

Edit: if I manually remove the lines

PRE_FLAGS+=( --gcc-install-dir=/opt/x86_64-linux-gnu/lib/gcc/x86_64-linux-gnu/6.1.0 )

in the cc/c++ wrappers then I can reproduce the C++ error above also on JuliaPackaging/BinaryBuilderBase.jl@65d6001, which means JuliaPackaging/BinaryBuilderBase.jl@e76628b is unrelated, it's only fixing the issue with the C compiler.

giordano · 2024-01-27T13:39:49Z

@gbaraldi I can confirm restoring --gcc-toolchain=/opt/x86_64-linux-gnu fixes the issue: JuliaPackaging/BinaryBuilderBase.jl#324 (comment).

Edit: fixed by JuliaPackaging/BinaryBuilderBase.jl#366.

giordano · 2024-01-27T16:33:39Z

@eschnett good to go now?

eschnett · 2024-01-27T18:54:17Z

@giordano Good to go.

* GMP: New version 6.3.0 * GMP: Don't use clang_use_lld * GMP: Update patches * GMP: Update patches for overflow handling * [GMP] Make copy of files verbose, to ease debugging --------- Co-authored-by: Mosè Giordano <[email protected]>

eschnett added 2 commits December 29, 2023 15:07

GMP: New version 6.3.0

b6101e2

GMP: Don't use clang_use_lld

eb56484

giordano reviewed Dec 29, 2023

View reviewed changes

G/GMP/[email protected]/build_tarballs.jl Outdated Show resolved Hide resolved

GMP: Update patches

13395fe

giordano marked this pull request as ready for review December 30, 2023 13:34

giordano marked this pull request as draft December 30, 2023 13:35

GMP: Update patches for overflow handling

7b4c058

eschnett marked this pull request as ready for review December 30, 2023 22:44

[GMP] Make copy of files verbose, to ease debugging

b95ede1

giordano force-pushed the eschnett/GMP-6.3.0 branch from 6d4d7f4 to b95ede1 Compare January 14, 2024 19:35

giordano mentioned this pull request Jan 27, 2024

[Runner] Fix MSAN C++ compiler with LLVM 13 JuliaPackaging/BinaryBuilderBase.jl#366

Merged

giordano marked this pull request as draft January 27, 2024 16:19

giordano marked this pull request as ready for review January 27, 2024 16:20

giordano merged commit 622d39e into JuliaPackaging:master Jan 27, 2024
25 checks passed

eschnett deleted the eschnett/GMP-6.3.0 branch January 27, 2024 19:15

ViralBShah mentioned this pull request May 31, 2024

[GMP] Add version 6.3.0 #7253

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GMP: New version 6.3.0 #7872

GMP: New version 6.3.0 #7872

eschnett commented Dec 29, 2023

eschnett commented Dec 30, 2023

eschnett commented Dec 30, 2023

maleadt commented Jan 1, 2024

giordano commented Jan 2, 2024

eschnett commented Jan 14, 2024

giordano commented Jan 14, 2024

eschnett commented Jan 15, 2024

giordano commented Jan 16, 2024

giordano commented Jan 27, 2024

giordano commented Jan 27, 2024 •

edited

Loading

giordano commented Jan 27, 2024 •

edited

Loading

giordano commented Jan 27, 2024 •

edited

Loading

giordano commented Jan 27, 2024

eschnett commented Jan 27, 2024

GMP: New version 6.3.0 #7872

GMP: New version 6.3.0 #7872

Conversation

eschnett commented Dec 29, 2023

eschnett commented Dec 30, 2023

eschnett commented Dec 30, 2023

maleadt commented Jan 1, 2024

giordano commented Jan 2, 2024

eschnett commented Jan 14, 2024

giordano commented Jan 14, 2024

eschnett commented Jan 15, 2024

giordano commented Jan 16, 2024

giordano commented Jan 27, 2024

giordano commented Jan 27, 2024 • edited Loading

giordano commented Jan 27, 2024 • edited Loading

giordano commented Jan 27, 2024 • edited Loading

giordano commented Jan 27, 2024

eschnett commented Jan 27, 2024

giordano commented Jan 27, 2024 •

edited

Loading

giordano commented Jan 27, 2024 •

edited

Loading

giordano commented Jan 27, 2024 •

edited

Loading