Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

overlaygen: cap max sparse file size #1294

Merged
merged 1 commit into from
Oct 31, 2024

Conversation

MichalHe
Copy link
Member

@MichalHe MichalHe commented Sep 5, 2024

If the source system has a very large disk attached, leapp would attempt to create similarly large sparse file, possibly hitting the max file size limits of the filesystem where these images are created. This patch caps the maximum size of any of the created sparse files to 1TB, preventing such issues.

Jira ref: RHEL-57064

Copy link

github-actions bot commented Sep 5, 2024

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

  • review please @oamg/developers to notify leapp developers of the review request
  • /packit copr-build to submit a public copr build using packit

Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build.
However, here are additional useful commands for packit:

  • /packit test to re-run manually the default tests
  • /packit retest-failed to re-run failed tests manually
  • /packit test oamg/leapp#42 to run tests with leapp builds for the leapp PR#42 (default is latest upstream - main - build)

Note that first time contributors cannot run tests automatically - they need to be started by a reviewer.

It is possible to schedule specific on-demand tests as well. Currently 2 test sets are supported, beaker-minimal and kernel-rt, both can be used to be run on all upgrade paths or just a couple of specific ones.
To launch on-demand tests with packit:

  • /packit test --labels kernel-rt to schedule kernel-rt tests set for all upgrade paths
  • /packit test --labels beaker-minimal-8.10to9.4,kernel-rt-8.10to9.4 to schedule kernel-rt and beaker-minimal test sets for 8.10->9.4 upgrade path

See other labels for particular jobs defined in the .packit.yaml file.

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra.

@MichalHe MichalHe force-pushed the limit_sparse_file_size branch from 7495b2d to 3bbd3e8 Compare September 5, 2024 20:16
Makefile Outdated
$(_PYTHON_VENV) -m pytest $(REPORT_ARG) $(TEST_PATHS) $(LIBRARY_PATH) $(PYTEST_ARGS)
$(_PYTHON_VENV) -m pytest $(REPORT_ARG) $(LIBRARY_PATH) $(PYTEST_ARGS)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I committed this by accident - if you do make test_no_lint you want to also test actors. I have removed the $(TEST_PATHS) because there was no way (at least obvious to me) to test just common libraries.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am actually puzzled now :-D because I checked the Makefile as the change looked weird to me originally but going through it, it started to make sense to me as a good step. Reading your explanation and the Makefile again, I am now trying to find out why I thought that :D
😶‍🌫️ + 🌴 --> 🤦

Even Homer nods 🤷

@pirat89 pirat89 added this to the 8.10/9.6 milestone Oct 25, 2024
On systems with large disks (e.g. 16TB) with lots of free space, leapp
might attemt to create files larger than the max file size of the
underlying FS. Attempting to create such large files causes leapp
to crash. This patch caps the max image size to 1TB, based on empirical
evidence that more free space is not needed for the upgrade RPM
transaction.

Jira-ref: RHEL-57064
@MichalHe MichalHe force-pushed the limit_sparse_file_size branch from 3bbd3e8 to 71c6248 Compare October 25, 2024 14:38
@pirat89 pirat89 self-assigned this Oct 29, 2024
@pirat89
Copy link
Member

pirat89 commented Oct 31, 2024

Tested. works as expected.

  • prepare machine
dd if=/dev/zero of=/root/tbdisk.img bs=1M count=0 seek=10875978  # ~11TB
mkfs.xfs  -l size=32m -f /root/tbdisk.img
mkdir /mnt/big_disk
echo >>/etc/fstab "/root/tbdisk.img /mnt/big_disk xfs loop 0 0"
systemctl daemon-reload
mount -a
  • original / upstream build (trying to create ~11TB img: 10260080 MiB)
# leapp-inspector actors --actor target_userspace_creator --terminal-like | grep -e "- \['/bin/dd'," | grep big_disk
    - ['/bin/dd', 'if=/dev/zero', 'of=/var/lib/leapp/scratch/diskimages/root_mnt_big_disk', 'bs=1M', 'count=0', 'seek=10260080']
  • PR build (trying to create just 1TB img: 1048576 MiB):
# leapp-inspector actors --actor target_userspace_creator --terminal-like | grep -e "- \['/bin/dd'," | grep big_disk
    - ['/bin/dd', 'if=/dev/zero', 'of=/var/lib/leapp/scratch/diskimages/root_mnt_big_disk', 'bs=1M', 'count=0', 'seek=1048576']

@pirat89 pirat89 added the bug Something isn't working label Oct 31, 2024
Copy link
Member

@pirat89 pirat89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm and works as expected (see comment above)

@pirat89 pirat89 merged commit cef2825 into oamg:main Oct 31, 2024
24 of 26 checks passed
@pirat89 pirat89 added the changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant label Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants