[BUG]: Unable to start when memory exceeds 27647

[BlackHole1]

After testing, when the --memory value is greater than 27647, krun will fail to start. The error is as follows:

Error: unable to begin running krun workload

[BlackHole1]

(27647 + 1) * 1024 * 1024 = 28,991,029,248

Memory max value is (27 GiB - 1M). Is this expected?

[tylerfanelli]

This is dependent on the system you're running on, no? 27GiB of RAM is no small amount. How much RAM is on the system you're running with?

[BlackHole1]

@tylerfanelli Sorry. I forgot to add my system information.

System: Darwin Arm64 (M3)
Memory: 36GB

[slp]

Works fine here with arbitrarily large memory sizes, even larger than the host's RAM (the memory for the guest is not preallocated). Could you please build krunkit with #18 and re-run your test with --krun-log-level 5?

[BlackHole1]

@slp Okay, I'll give it a try now.

[BlackHole1]

LOG:

[2024-11-04T11:10:40Z DEBUG vmm::macos::vstate] Guest memory host_addr=0x104d44000 guest_addr=0 len=8000000
[2024-11-04T11:10:40Z DEBUG vmm::macos::vstate] Guest memory host_addr=0x300000000 guest_addr=40000000 len=6c0000000
[2024-11-04T11:10:40Z DEBUG vmm::macos::vstate] Guest memory host_addr=0x7000000000 guest_addr=740000000 len=900000000
[2024-11-04T11:10:40Z ERROR krun] Building the microVM failed: Internal(Vm(SetUserMemoryRegion(MemoryMap)))
Error: unable to begin running krun workload

[BlackHole1]

I found that orbstack also encountered a similar issue, but since it is closed-source software, I am not sure how it was fixed: orbstack/orbstack#1178

[BlackHole1]

I added some logs to libkrun/hvf/src/lib.rs#L201 and found that the return value of hv_vm_map is: HV_BAD_ARGUMENT.

HV_BAD_ARGUMENT: https://developer.apple.com/documentation/hypervisor/1585168-hypervisor_errors/hv_bad_argument

[BlackHole1]

Relevant discussion:

• VM Startup (QEMU RAM Allocation) Errors with High (>32GB) RAM on M2 Max Macbook utmapp/UTM#5537
• Machine memory limit with podman under QEMU on MacOS and Apple Silicon podman#19528

[slp]

Could you please give #19 a try? I think it should work with arbitrary memory configurations, even ones exceeding the memory available on the host (with the upper bound of 60 GiB).

[BlackHole1]

@slp It worked! Thanks!