Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VM] Override address size limit to unlimited #827

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Ionic
Copy link
Contributor

@Ionic Ionic commented Jun 3, 2022

The "build" script limits PRLIMIT_AS to (2/3) * (mem_size + swap_size),
which breaks asan, which tries to opportunistically allocate multiple TB
of memory, but crucially never even closely uses this.

This commit eliminates the memory limit right before spawning VMs (and
containers) if necessary, so that stage2 can run unrestricted
(notwithstanding the memory limit imposed by hypervisors) and use memory
overcommitment within the build process.

See the linked bug for a discussion of this.

Fixes: #826

The "build" script limits PRLIMIT_AS to (2/3) * (mem_size + swap_size),
which breaks asan, which tries to opportunistically allocate multiple TB
of memory, but crucially never even closely uses this.

This commit eliminates the memory limit right before spawning VMs (and
containers) if necessary, so that stage2 can run unrestricted
(notwithstanding the memory limit imposed by hypervisors) and use memory
overcommitment within the build process.

See the linked bug for a discussion of this.

Fixes: openSUSE#826
@adrianschroeter
Copy link
Member

the ulimit calls would fail when using build script as non-root?

I agree that a limit when doing VM builds seem not to be necessary, not sure about this approach though.

@dirkmueller
Copy link
Member

is this still needed after 52cec99 ?

@Ionic
Copy link
Contributor Author

Ionic commented Aug 16, 2022

the ulimit calls would fail when using build script as non-root?

Probably, yes, but doesn't spawning a build require root access to begin with? The actual build process is being run unprivileged in most cases, but the setup typically requires elevated privileges.

is this still needed after 52cec99 ?

Very good question. That one (obviously) is completely new to me. Looking at it and reading up, it does look like my PR is obsolete after the mentioned commit and that it fixes asan usage within VMs/containers. I'll test that and close the PR and bug report if it turns out to essentially allow memory overcommitment.

@Ionic
Copy link
Contributor Author

Ionic commented Sep 2, 2022

Unfortunately, even 52cec99 breaks address sanitizers with LXC builds.

After that change, I'll have to override lxc.prlimit.data to get packages with address sanitizers working, of course, but it essentially boils down to the same breakage.

I can't really test the other virtualization/containerization options (I'd prefer using Qemu/KVM in the first place, but the VM I'm running an OBS appliance on doesn't support nested virtualization, so I'm out of luck on that front).

Will update this PR accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[RFC] Support address sanitizers within VMs via memory overcommit
3 participants