Add icb configurable defrag inflight #13238

robdrynkin · 2025-01-09T15:00:17Z

Not for changelog (changelog entry is not required)

github-actions · 2025-01-09T15:05:28Z

⚪ 2025-01-09 15:05:27 UTC Pre-commit check linux-x86_64-relwithdebinfo for 3351c62 has started.
⚪ 2025-01-09 15:05:39 UTC Artifacts will be uploaded here
⚪ 2025-01-09 15:08:49 UTC ya make is running...
🟡 2025-01-09 16:32:13 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
27463	24953	0	6	2383	121

⚪ 2025-01-09 16:35:10 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-01-09 16:47:03 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
183 (only retried tests)	63	0	1	0	119

⚪ 2025-01-09 16:47:11 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-01-09 16:59:03 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
174 (only retried tests)	55	0	1	0	118

🟢 2025-01-09 16:59:09 UTC Build successful.
🟢 2025-01-09 16:59:31 UTC ydbd size 2.1 GiB changed* by +5.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `f86b5e4`	merge: `3351c62`	diff	diff %
ydbd size	2 244 703 744 Bytes	2 244 709 120 Bytes	+5.2 KiB	+0.000%
ydbd stripped size	472 407 312 Bytes	472 408 080 Bytes	+768 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-01-09T15:05:40Z

⚪ 2025-01-09 15:05:39 UTC Pre-commit check linux-x86_64-release-asan for 3351c62 has started.
⚪ 2025-01-09 15:05:52 UTC Artifacts will be uploaded here
⚪ 2025-01-09 15:09:03 UTC ya make is running...
🟡 2025-01-09 16:33:29 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13460	13397	0	27	7	29

⚪ 2025-01-09 16:34:48 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-01-09 16:46:45 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
107 (only retried tests)	75	0	6	0	26

⚪ 2025-01-09 16:46:53 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-01-09 16:58:37 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
59 (only retried tests)	30	0	4	1	24

🟢 2025-01-09 16:58:45 UTC Build successful.
🟢 2025-01-09 16:59:14 UTC ydbd size 3.6 GiB changed* by +16.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `f86b5e4`	merge: `3351c62`	diff	diff %
ydbd size	3 874 445 360 Bytes	3 874 461 744 Bytes	+16.0 KiB	+0.000%
ydbd stripped size	1 350 832 368 Bytes	1 350 838 896 Bytes	+6.4 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

va-kuznecov · 2025-01-09T17:36:03Z

ydb/core/protos/config.proto

+            Description: "Max inflight defragmentation chunks per one chunk size per vdisk",
+            MinValue: 1,
+            MaxValue: 50,
+            DefaultValue: 1 }];


Suggested change

DefaultValue: 1 }];

DefaultValue: 10 }];

va-kuznecov · 2025-01-09T17:36:11Z

ydb/core/blobstorage/nodewarden/node_warden_impl.cpp

@@ -30,6 +30,7 @@ TNodeWarden::TNodeWarden(const TIntrusivePtr<TNodeWardenConfig> &cfg)
    , MaxSyncLogChunksInFlightSSD(10, 1, 1024)
    , DefaultHugeGarbagePerMille(300, 1, 1000)
    , HugeDefragFreeSpaceBorderPerMille(260, 1, 1000)
+    , MaxChunksToDefragInflight(1, 1, 50)


Suggested change

, MaxChunksToDefragInflight(1, 1, 50)

, MaxChunksToDefragInflight(10, 1, 50)

github-actions · 2025-01-14T15:12:06Z

⚪ 2025-01-14 15:12:05 UTC Pre-commit check linux-x86_64-release-asan for 67706d7 has started.
⚪ 2025-01-14 15:12:17 UTC Artifacts will be uploaded here
⚪ 2025-01-14 15:15:27 UTC ya make is running...
🔴 2025-01-14 15:33:50 UTC Build failed, see the logs. Also see fail summary

github-actions · 2025-01-14T15:12:13Z

⚪ 2025-01-14 15:12:12 UTC Pre-commit check linux-x86_64-relwithdebinfo for 67706d7 has started.
⚪ 2025-01-14 15:12:25 UTC Artifacts will be uploaded here
⚪ 2025-01-14 15:15:47 UTC ya make is running...
🔴 2025-01-14 15:30:01 UTC Build failed, see the logs. Also see fail summary

github-actions · 2025-01-15T14:27:33Z

⚪ 2025-01-15 14:27:32 UTC Pre-commit check linux-x86_64-relwithdebinfo for 24d6963 has started.
⚪ 2025-01-15 14:27:40 UTC Artifacts will be uploaded here
⚪ 2025-01-15 14:30:56 UTC ya make is running...
🟡 2025-01-15 15:57:07 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
27524	25016	0	2	2380	126

⚪ 2025-01-15 16:00:13 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-01-15 16:12:36 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
182 (only retried tests)	63	0	0	0	119

🟢 2025-01-15 16:12:43 UTC Build successful.
🟢 2025-01-15 16:13:06 UTC ydbd size 2.1 GiB changed* by +5.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `6bcf4a6`	merge: `24d6963`	diff	diff %
ydbd size	2 251 972 592 Bytes	2 251 977 984 Bytes	+5.3 KiB	+0.000%
ydbd stripped size	473 567 248 Bytes	473 568 016 Bytes	+768 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-01-15T14:29:14Z

⚪ 2025-01-15 14:29:13 UTC Pre-commit check linux-x86_64-release-asan for 24d6963 has started.
⚪ 2025-01-15 14:29:25 UTC Artifacts will be uploaded here
⚪ 2025-01-15 14:32:57 UTC ya make is running...
🟡 2025-01-15 16:09:08 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13497	13436	0	19	15	27

⚪ 2025-01-15 16:10:26 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-01-15 16:22:30 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
103 (only retried tests)	77	0	0	0	26

🟢 2025-01-15 16:22:37 UTC Build successful.
🟢 2025-01-15 16:23:04 UTC ydbd size 3.6 GiB changed* by +12.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `6bcf4a6`	merge: `24d6963`	diff	diff %
ydbd size	3 903 127 776 Bytes	3 903 140 096 Bytes	+12.0 KiB	+0.000%
ydbd stripped size	1 362 986 384 Bytes	1 362 988 752 Bytes	+2.3 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

ydb/core/blobstorage/vdisk/common/vdisk_config.h

github-actions · 2025-01-15T17:54:47Z

⚪ 2025-01-15 17:54:46 UTC Pre-commit check linux-x86_64-relwithdebinfo for 5bbddb0 has started.
⚪ 2025-01-15 17:54:57 UTC Artifacts will be uploaded here
⚪ 2025-01-15 17:58:18 UTC ya make is running...
🟡 2025-01-15 19:25:53 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
27533	25021	0	3	2386	123

⚪ 2025-01-15 19:28:31 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-01-15 19:40:49 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
188 (only retried tests)	70	0	0	0	118

🟢 2025-01-15 19:40:57 UTC Build successful.
🟢 2025-01-15 19:41:18 UTC ydbd size 2.1 GiB changed* by +5.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `ca6eab5`	merge: `5bbddb0`	diff	diff %
ydbd size	2 252 624 848 Bytes	2 252 630 240 Bytes	+5.3 KiB	+0.000%
ydbd stripped size	473 493 520 Bytes	473 494 288 Bytes	+768 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-01-15T17:55:28Z

⚪ 2025-01-15 17:55:27 UTC Pre-commit check linux-x86_64-release-asan for 5bbddb0 has started.
⚪ 2025-01-15 17:55:31 UTC Artifacts will be uploaded here
⚪ 2025-01-15 17:58:41 UTC ya make is running...
🟡 2025-01-15 19:26:01 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13506	13451	0	21	9	25

⚪ 2025-01-15 19:27:21 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-01-15 19:39:32 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
86 (only retried tests)	59	0	0	3	24

🟢 2025-01-15 19:39:39 UTC Build successful.
🟢 2025-01-15 19:40:05 UTC ydbd size 3.6 GiB changed* by +12.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `ca6eab5`	merge: `5bbddb0`	diff	diff %
ydbd size	3 903 073 088 Bytes	3 903 085 408 Bytes	+12.0 KiB	+0.000%
ydbd stripped size	1 362 738 288 Bytes	1 362 740 656 Bytes	+2.3 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

Co-authored-by: Vlad Kuznetsov <[email protected]>

Add icb configurable defrag inflight

8511d8d

github-actions bot added the not-for-changelog label Jan 9, 2025

va-kuznecov reviewed Jan 9, 2025

View reviewed changes

robdrynkin added 2 commits January 13, 2025 16:16

Change default

282147a

Fix test

d357437

Fix build

4c01138

va-kuznecov reviewed Jan 15, 2025

View reviewed changes

ydb/core/blobstorage/vdisk/common/vdisk_config.h Outdated Show resolved Hide resolved

Update ydb/core/blobstorage/vdisk/common/vdisk_config.h

08d9202

va-kuznecov approved these changes Jan 15, 2025

View reviewed changes

robdrynkin merged commit b01ebea into ydb-platform:main Jan 16, 2025
12 checks passed

robdrynkin added a commit to robdrynkin/ydb that referenced this pull request Jan 17, 2025

Add icb configurable defrag inflight (ydb-platform#13238)

c0fc6a0

Co-authored-by: Vlad Kuznetsov <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add icb configurable defrag inflight #13238

Add icb configurable defrag inflight #13238

robdrynkin commented Jan 9, 2025

github-actions bot commented Jan 9, 2025 •

edited

Loading

github-actions bot commented Jan 9, 2025 •

edited

Loading

va-kuznecov Jan 9, 2025

va-kuznecov Jan 9, 2025

github-actions bot commented Jan 14, 2025 •

edited

Loading

github-actions bot commented Jan 14, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

	, MaxChunksToDefragInflight(1, 1, 50)
	, MaxChunksToDefragInflight(10, 1, 50)

Add icb configurable defrag inflight #13238

Add icb configurable defrag inflight #13238

Conversation

robdrynkin commented Jan 9, 2025

github-actions bot commented Jan 9, 2025 • edited Loading

github-actions bot commented Jan 9, 2025 • edited Loading

va-kuznecov Jan 9, 2025

Choose a reason for hiding this comment

va-kuznecov Jan 9, 2025

Choose a reason for hiding this comment

github-actions bot commented Jan 14, 2025 • edited Loading

github-actions bot commented Jan 14, 2025 • edited Loading

github-actions bot commented Jan 15, 2025 • edited Loading

github-actions bot commented Jan 15, 2025 • edited Loading

github-actions bot commented Jan 15, 2025 • edited Loading

github-actions bot commented Jan 15, 2025 • edited Loading

github-actions bot commented Jan 9, 2025 •

edited

Loading

github-actions bot commented Jan 9, 2025 •

edited

Loading

github-actions bot commented Jan 14, 2025 •

edited

Loading

github-actions bot commented Jan 14, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading