Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpmbuilds: dynamically specify tmp_fs size #3408

Merged
merged 2 commits into from
Sep 25, 2024

Conversation

nikromen
Copy link
Member

@nikromen nikromen commented Sep 9, 2024

This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

before merge (please review):

Fixes #3268

rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
@nikromen
Copy link
Member Author

nikromen commented Sep 9, 2024

"deployed" on dev and works

powerful:
https://copr.stg.fedoraproject.org/coprs/g/copr/powerful-test/build/2918769/

normal:
https://copr.stg.fedoraproject.org/coprs/nikromen/not-pulp/build/2918768/

powerful builders have 320GB volume (304 swap). (how do I confirm the max_fs size of 280gb?)

@FrostyX
Copy link
Member

FrostyX commented Sep 10, 2024

LGTM. The only thing worth pointing out is the hardcoded volume size numbers but I think it's fine.

self.tags = task.get("tags", [])
self.max_fs_size = "140g"
if "on_demand_powerful" in self.tags:
self.max_fs_size = "280g"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not against having (some) default, but in 99% this is deployment-specific info. And 140G is typically quite too much for local reproducibility (the user tries to run the copr-rpmbuild locally to reproduce a build failure, e.g.).

@praiskup
Copy link
Member

praiskup commented Sep 10, 2024

powerful builders have 320GB volume (304 swap). (how do I confirm the max_fs size of 280gb?)

On my system I run: mount | grep mock

mock_chroot_tmpfs on /var/lib/mock/fedora-40-x86_64-bootstrap/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)
mock_chroot_tmpfs on /var/lib/mock/fedora-40-x86_64/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)
mock_chroot_tmpfs on /var/lib/mock/fedora-rawhide-ppc64le-bootstrap/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)
mock_chroot_tmpfs on /var/lib/mock/fedora-rawhide-ppc64le/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)
mock_chroot_tmpfs on /var/lib/mock/fedora-39-ppc64le-bootstrap/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)
mock_chroot_tmpfs on /var/lib/mock/fedora-41-x86_64-bootstrap/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)
mock_chroot_tmpfs on /var/lib/mock/fedora-39-ppc64le/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)
mock_chroot_tmpfs on /var/lib/mock/fedora-41-x86_64/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)
mock_chroot_tmpfs on /var/lib/mock/fedora-38-ppc64le-bootstrap/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)
mock_chroot_tmpfs on /var/lib/mock/fedora-38-ppc64le/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)
mock_chroot_tmpfs on /var/lib/mock/rhel+epel-9-ppc64le-bootstrap/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)
mock_chroot_tmpfs on /var/lib/mock/rhel+epel-9-ppc64le/root type tmpfs (rw,relatime,seclabel,size=146800640k,nr_inodes=0,mode=755,inode64)

I believe the size=* mount argument is related.

@praiskup
Copy link
Member

update resalloc cloud providers - fedora-copr/resalloc-ibm-cloud#28

Not needed right now IMVHO, or has s390x support been requested?

rpmbuild/copr_rpmbuild/config.py Fixed Show fixed Hide fixed
rpmbuild/copr_rpmbuild/config.py Fixed Show fixed Hide fixed
rpmbuild/copr_rpmbuild/config.py Fixed Show fixed Hide fixed
rpmbuild/copr_rpmbuild/config.py Fixed Show fixed Hide fixed
rpmbuild/copr_rpmbuild/config.py Fixed Show fixed Hide fixed
rpmbuild/copr_rpmbuild/providers/base.py Fixed Show fixed Hide fixed
rpmbuild/main.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
@nikromen
Copy link
Member Author

Not needed right now IMVHO, or has s390x support been requested?

this needs to be done for every arch that has on_demand_powerful supported - so even s390x

@praiskup
Copy link
Member

this needs to be done for every arch that has on_demand_powerful supported - so even s390x

Not really, we can have different configurations for different architectures; but it seems the original requestor wanted to have support for all the architectures.

@praiskup
Copy link
Member

Please provide the configuration file as %config(noreplace) in the RPM file,
and self-document the option. Something like:

---
# Configure special mock configuration snippets per given set of tags.
#tags_to_mock_snippets:
#  - tags:
#      - on_demand_powerful
#      - arch_x86_64
#    snippet: |
#        config_opts['plugin_conf']['tmpfs_opts']['max_fs_size'] = '140g'

@nikromen nikromen force-pushed the more-storage-builders branch 3 times, most recently from b45b43a to 34455a8 Compare September 17, 2024 11:44
rpmbuild/tests/__init__.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
@nikromen nikromen force-pushed the more-storage-builders branch 5 times, most recently from 2d5ca8a to e07b957 Compare September 17, 2024 16:44
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
rpmbuild/tests/test_mock.py Fixed Show fixed Hide fixed
@nikromen nikromen force-pushed the more-storage-builders branch 3 times, most recently from 3d2e928 to 48fd69e Compare September 17, 2024 17:06
@praiskup
Copy link
Member

This is in a good shape, thank you for the update! Two small things that would make it perfect :) but basically good to go.

This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
@praiskup
Copy link
Member

Fun -> I thought we had broken Fedora Copr! But the testsuite hangs on tests/test_base.py::TestProvider::test_create_rpmmacros for some reason (and eats more and more memory till it fails).

@nikromen nikromen merged commit 8b0977a into fedora-copr:main Sep 25, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a pool of builders that have more storage
3 participants