Segmentation fault for large lattice on Apple M1

[bachpc]

I encountered segfault at https://github.com/keeganryan/flatter/blob/main/src/data/lattice/lattice.cpp#L104 on my M1, but this issue never appeared when I run on my Ubuntu WSL.

    for (unsigned int i = 0; i < dim; i++) {
        for (unsigned int j = 0; j < rank; j++) {
            mpz_set_str(dM(i, j), data[j][i].c_str(), 0);
        }
    }

Input lattice: example_lattice.txt (40 x 40 lattice with ~8100 digits per entry)

CMake log:

-- The CXX compiler identification is Clang 15.0.7
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Users/lap15281/mambaforge/envs/sage/bin/arm64-apple-darwin20.0.0-clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_CXX: -fopenmp=libomp (found version "5.0") 
-- Found OpenMP: TRUE (found version "5.0")  
-- Found GMP: /Users/lap15281/mambaforge/envs/sage/include  
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: /Users/lap15281/mambaforge/envs/sage/lib/libopenblas.dylib  
-- Looking for cheev_
-- Looking for cheev_ - found
-- Found LAPACK: /Users/lap15281/mambaforge/envs/sage/lib/libopenblas.dylib;-lm;-ldl  
-- Found MPFR: /Users/lap15281/mambaforge/envs/sage/include (Required is at least version "1.0.0") 
-- Found FPLLL: /Users/lap15281/mambaforge/envs/sage/include (found suitable version "5.4.4", minimum required is "5.1.0") 
-- Configuring done (2.8s)
-- Generating done (0.1s)
-- Build files have been written to: /Users/lap15281/Desktop/Workspace/flatter/build

[keeganryan]

I've tried reproducing this issue, but I've been unable to. Does flatter work on M1 with smaller lattice bases? If so, then it's possibly a memory allocation issue, and flatter isn't handling allocation errors gracefully.

It looks like it's segfaulting early in the process as it reads the lattice from a file into memory. This may be one of the first calls into libgmp. Another possibility is a mismatch between the version of GMP the program is linked against and the version it is run against, which may possibly happen if there's another version of GMP outside of sage.

[bachpc]

I have a mambaforge sage environment that includes installations of SageMath, GMP, FPLLL, and MPFR. It appears that the dependencies detected by CMake are specific to this environment. Can you able to reproduce with that?

Flatter works well with smaller lattice bases, but it encounters segfault with larger lattice bases. And it seems that flatter is capable of loading some initial entries using mpz_set_str before encountering a segfault.