Crash on allocation failure

[pitrou]

I'm trying to move Apache Arrow to mimalloc v3.1.5 but we're hitting a crash in one of our tests that tries to allocate a very large memory area. It should ideally fail gracefully instead.

Here is the test log when running under gdb:

[----------] 4 tests from Default/TestMemoryPool/0, where TypeParam = arrow::DefaultMemoryPoolFactory
[ RUN      ] Default/TestMemoryPool/0.MemoryTracking
[       OK ] Default/TestMemoryPool/0.MemoryTracking (0 ms)
[ RUN      ] Default/TestMemoryPool/0.OOM
mimalloc: error: aligned allocation request is too large (size 9223372036854775807, alignment 64)
mimalloc: warning: unable to allocate OS memory (error: 12 (0x0C), addr: 0x00000000, size: 0x8000000000810000 bytes, align: 0x01, commit: 1, allow large: 0)

Thread 1 "arrow-misc-test" received signal SIGSEGV, Segmentation fault.
0x00007ffff78bebfb in mi_page_start (page=0x0) at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/include/mimalloc/internal.h:654
/home/antoine/mambaforge/envs/pyarrow/lib/python3.10/site-packages/astroid/interpreter/_import/util.py:4: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
654	  return page->page_start;

Here is the gdb backtrace:

(gdb) bt
#0  0x00007ffff78bebfb in mi_page_start (page=0x0) at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/include/mimalloc/internal.h:654
#1  0x00007ffff78c1d1b in _mi_arenas_page_alloc (heap=0x7ffff7931300 <heap_main>, block_size=9223372036858970112, block_alignment=0)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/arena.c:803
#2  0x00007ffff78d517c in mi_page_fresh_alloc (heap=0x7ffff7931300 <heap_main>, pq=0x7ffff7932118 <heap_main+3608>, block_size=9223372036858970112, page_alignment=0)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/page.c:305
#3  0x00007ffff78d6b12 in mi_huge_page_alloc (heap=0x7ffff7931300 <heap_main>, size=9223372036854775815, page_alignment=0, pq=0x7ffff7932118 <heap_main+3608>)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/page.c:897
#4  0x00007ffff78d6d0c in mi_find_page (heap=0x7ffff7931300 <heap_main>, size=9223372036854775815, huge_alignment=0)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/page.c:925
#5  0x00007ffff78d6ef7 in _mi_malloc_generic (heap=0x7ffff7931300 <heap_main>, size=9223372036854775815, zero=false, huge_alignment=0)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/page.c:968
#6  0x00007ffff78bd404 in _mi_heap_malloc_zero_ex (heap=0x7ffff7931300 <heap_main>, size=9223372036854775807, zero=false, huge_alignment=0)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/alloc.c:187
#7  0x00007ffff78bd481 in _mi_heap_malloc_zero (heap=0x7ffff7931300 <heap_main>, size=9223372036854775807, zero=false)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/alloc.c:200
#8  0x00007ffff78bdbaf in mi_heap_malloc_zero_no_guarded (heap=0x7ffff7931300 <heap_main>, size=9223372036854775807, zero=false)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/alloc-aligned.c:51
#9  0x00007ffff78bdd2d in mi_heap_malloc_zero_aligned_at_overalloc (heap=0x7ffff7931300 <heap_main>, size=9223372036854775744, alignment=64, offset=0, zero=false)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/alloc-aligned.c:82
#10 0x00007ffff78be120 in mi_heap_malloc_zero_aligned_at_generic (heap=0x7ffff7931300 <heap_main>, size=9223372036854775744, alignment=64, offset=0, zero=false)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/alloc-aligned.c:164
#11 0x00007ffff78be354 in mi_heap_malloc_zero_aligned_at (heap=0x7ffff7931300 <heap_main>, size=9223372036854775744, alignment=64, offset=0, zero=false)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/alloc-aligned.c:206
#12 0x00007ffff78be38c in mi_heap_malloc_aligned_at (heap=0x7ffff7931300 <heap_main>, size=9223372036854775744, alignment=64, offset=0)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/alloc-aligned.c:215
#13 0x00007ffff78be3bb in mi_heap_malloc_aligned (heap=0x7ffff7931300 <heap_main>, size=9223372036854775744, alignment=64)
    at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/alloc-aligned.c:219
#14 0x00007ffff78be3e8 in mi_malloc_aligned (size=9223372036854775744, alignment=64) at /build/build-test/mimalloc_ep-prefix/src/mimalloc_ep/src/alloc-aligned.c:249
#15 0x00007ffff7633667 in arrow::(anonymous namespace)::MimallocAllocator::AllocateAligned (size=9223372036854775744, alignment=64, out=0x7fffffffbc50)
    at /home/antoine/arrow/dev/cpp/src/arrow/memory_pool.cc:404
[ ... snip ...]

[pitrou]

The problem is that _mi_arenas_page_alloc does not check that page was allocated successfully before making assertions about page_start(page). @daanx

[malkia]

@pitrou - Thanks (we've just rolled mimalloc 3.1.5 internally, so I was on the lookout for issues). So this would crash because it'll call several mi_assert_internal (e.g. in secure or debug mode), and then effectively do the same as your patch. Thanks btw! I'll integrate this into our branch!