feat: Use fallocate for file size extension when supported #11541
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for meta-velox canceled.
|
zacw7
force-pushed
the
fallocate
branch
2 times, most recently
from
83b7711 to
4bed29d
Compare
zacw7
changed the title
|
@zacw7 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@zacw7 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
xiaoxmeng
reviewed
Nov 19, 2024
|
|
||
| namespace facebook::velox::cache::test { | ||
|
|
||
| class SsdFileTestHelper { |
There was a problem hiding this comment.
This needs to be a header file
zacw7
added a commit
to zacw7/velox
that referenced
this pull request
Nov 20, 2024
…ncubator#11541) Summary: When Copy-on-Write (COW) is disabled on Btrfs, automatic relocation creates snapshots of files, ignoring the noCOW setting. This results in increased disk usage and can lead to "no space left" errors in production. One possible enhancement we can make is to use fallocate to reserve space immediately after file creation. This helps ensure the allocated space is as continuous as possible. For environments where "no space left" errors are already happening, attempting to reserve space can result in failures. In that case, fallback to truncate and record it in runtime metrics. Differential Revision: D65977522 Pulled By: zacw7
|
This pull request was exported from Phabricator. Differential Revision: D65977522 |
xiaoxmeng
approved these changes
Nov 20, 2024
| namespace test { | ||
| class SsdCacheTestHelper; | ||
| } | ||
|
|
||
| class SsdCache { |
There was a problem hiding this comment.
Can you remove all existing testing methods from the two classes with test helper? thanks!
There was a problem hiding this comment.
Let's do it in a follow up PR?
zacw7
added a commit
to zacw7/velox
that referenced
this pull request
Nov 20, 2024
…ncubator#11541) Summary: When Copy-on-Write (COW) is disabled on Btrfs, automatic relocation creates snapshots of files, ignoring the noCOW setting. This results in increased disk usage and can lead to "no space left" errors in production. One possible enhancement we can make is to use fallocate to reserve space immediately after file creation. This helps ensure the allocated space is as continuous as possible. For environments where "no space left" errors are already happening, attempting to reserve space can result in failures. In that case, fallback to truncate and record it in runtime metrics. Reviewed By: xiaoxmeng Differential Revision: D65977522 Pulled By: zacw7
|
This pull request was exported from Phabricator. Differential Revision: D65977522 |
zacw7
added a commit
to zacw7/velox
that referenced
this pull request
Nov 20, 2024
…ncubator#11541) Summary: When Copy-on-Write (COW) is disabled on Btrfs, automatic relocation creates snapshots of files, ignoring the noCOW setting. This results in increased disk usage and can lead to "no space left" errors in production. One possible enhancement we can make is to use fallocate to reserve space immediately after file creation. This helps ensure the allocated space is as continuous as possible. For environments where "no space left" errors are already happening, attempting to reserve space can result in failures. In that case, fallback to truncate and record it in runtime metrics. Differential Revision: D65977522 Pulled By: zacw7
|
This pull request was exported from Phabricator. Differential Revision: D65977522 |
…ncubator#11541) Summary: When Copy-on-Write (COW) is disabled on Btrfs, automatic relocation creates snapshots of files, ignoring the noCOW setting. This results in increased disk usage and can lead to "no space left" errors in production. One possible enhancement we can make is to use fallocate to reserve space immediately after file creation. This helps ensure the allocated space is as continuous as possible. For environments where "no space left" errors are already happening, attempting to reserve space can result in failures. In that case, fallback to truncate and record it in runtime metrics. Reviewed By: xiaoxmeng Differential Revision: D65977522 Pulled By: zacw7
|
This pull request was exported from Phabricator. Differential Revision: D65977522 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When Copy-on-Write (COW) is disabled on Btrfs, automatic relocation creates snapshots of files, ignoring the noCOW setting. This results in increased disk usage and can lead to "no space left" errors in production.
One possible enhancement we can make is to use fallocate to reserve space immediately after file creation. This helps ensure the allocated space is as continuous as possible.
For environments where "no space left" errors are already happening, attempting to reserve space can result in failures. In that case, fallback to truncate and record it in runtime metrics.