Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write child compaction wiki #6

Open
rjl493456442 opened this issue Oct 19, 2020 · 0 comments
Open

Write child compaction wiki #6

rjl493456442 opened this issue Oct 19, 2020 · 0 comments

Comments

@rjl493456442
Copy link
Member

Child compaction is another killer feature of go-leveldb project.

Although the concurrent compaction is already enabled by default
which speeds up the compaction efficiency a lot, but the write stall
is still not fixed.

What's write stall

Write stall refers to the phenomenon that the system write is disabled
temporarily because of the SLOW COMPACTION. The root issue is
the level0 compaction is not fast enough.

But currently the concurrent level0 compaction is not feasible because
the files in level0 are overlapped with each other. In this case we need
some additional mechanism to improve the efficiency of level0 compaction.
I would say the efficiency of level0 compaction is the KEY of the leveldb
write performance.

Pebble has the idea called SubLevel0Compaction. The idea is quite
straightforward: create indexes for level0 files whenver the level0 content
is changed, try to schedule concurrent level0 compactions for these
"non-overlapped" files. However in practice, it's not so easy to find such
files.

So we had this idea called ChildCompaction. Let's write it down for further
discussion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant