Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify scheduler for compaction/file purge #2153

Closed
v0y4g3r opened this issue Aug 11, 2023 · 0 comments · Fixed by #2194
Closed

Simplify scheduler for compaction/file purge #2153

v0y4g3r opened this issue Aug 11, 2023 · 0 comments · Fixed by #2194
Assignees
Labels
C-enhancement Category Enhancements

Comments

@v0y4g3r
Copy link
Contributor

v0y4g3r commented Aug 11, 2023

What type of enhancement is this?

Refactor

What does the enhancement do?

Currently the Scheduler trait uses an associate type Request,

#[async_trait]
pub trait Scheduler: Debug {
    type Request;

    fn schedule(&self, request: Self::Request) -> Result<bool>;

    async fn stop(&self, await_termination: bool) -> Result<()>;
}

and everytime we add a new scheduler we have to implement all the associate structs like Request/Handler, etc.

This design aims to reduce polymorphic overhead leveraging static dispatching. But this overhead is negligible in this case.
We can use a trait object to simplify the code.

Implementation challenges

No response

@v0y4g3r v0y4g3r added the C-enhancement Category Enhancements label Aug 11, 2023
@evenyag evenyag added this to mito2 Aug 14, 2023
@evenyag evenyag moved this to In Progress in mito2 Aug 14, 2023
github-merge-queue bot pushed a commit that referenced this issue Aug 23, 2023
)

* feature: add a simple scheduler using flume

Signed-off-by: ZhuZiyi <[email protected]>

* fix: only use a sender rather clone many senders

Signed-off-by: ZhuZiyi <[email protected]>

* fix: use select to avoid loop

Signed-off-by: ZhuZiyi <[email protected]>

* feat: add parameters in new function to build the flume capacity and number of receivers

Signed-off-by: ZhuZiyi <[email protected]>

* test: add countdownlatch test concurrency

Signed-off-by: ZhuZiyi <[email protected]>

* test: add barrier replacing countdownlatch to test concurrency and add wait all tasks finished in stop

Signed-off-by: ZhuZiyi <[email protected]>

* chore: add some document annotation

Signed-off-by: ZhuZiyi <[email protected]>

* chore: add license header

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* chore: add Cargo.lock

Signed-off-by: ZhuZiyi <[email protected]>

* chore: Cargo.toml format

Signed-off-by: ZhuZiyi <[email protected]>

* chore: delete println in test

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* feat: add error handle

Signed-off-by: ZhuZiyi <[email protected]>

* fix: fix error handle and add test scheduler stop

Signed-off-by: ZhuZiyi <[email protected]>

* chore: spelling mistake

Signed-off-by: ZhuZiyi <[email protected]>

* fix: wait all tasks finished

Signed-off-by: ZhuZiyi <[email protected]>

* chore: add todo which need wrap Future returned by send_async

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* test: remove unnessary sleep in test

Signed-off-by: ZhuZiyi <[email protected]>

* fix: resolve some conflicts

Signed-off-by: ZhuZiyi <[email protected]>

* fix: resolve conversation

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* feat: modify the function of schedule to synchronize and drop sender after stopping scheduler

Signed-off-by: ZhuZiyi <[email protected]>

---------

Signed-off-by: ZhuZiyi <[email protected]>
@github-project-automation github-project-automation bot moved this from In Progress to Done in mito2 Aug 23, 2023
paomian pushed a commit to paomian/greptimedb that referenced this issue Oct 19, 2023
…eam#2153 (GreptimeTeam#2194)

* feature: add a simple scheduler using flume

Signed-off-by: ZhuZiyi <[email protected]>

* fix: only use a sender rather clone many senders

Signed-off-by: ZhuZiyi <[email protected]>

* fix: use select to avoid loop

Signed-off-by: ZhuZiyi <[email protected]>

* feat: add parameters in new function to build the flume capacity and number of receivers

Signed-off-by: ZhuZiyi <[email protected]>

* test: add countdownlatch test concurrency

Signed-off-by: ZhuZiyi <[email protected]>

* test: add barrier replacing countdownlatch to test concurrency and add wait all tasks finished in stop

Signed-off-by: ZhuZiyi <[email protected]>

* chore: add some document annotation

Signed-off-by: ZhuZiyi <[email protected]>

* chore: add license header

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* chore: add Cargo.lock

Signed-off-by: ZhuZiyi <[email protected]>

* chore: Cargo.toml format

Signed-off-by: ZhuZiyi <[email protected]>

* chore: delete println in test

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* feat: add error handle

Signed-off-by: ZhuZiyi <[email protected]>

* fix: fix error handle and add test scheduler stop

Signed-off-by: ZhuZiyi <[email protected]>

* chore: spelling mistake

Signed-off-by: ZhuZiyi <[email protected]>

* fix: wait all tasks finished

Signed-off-by: ZhuZiyi <[email protected]>

* chore: add todo which need wrap Future returned by send_async

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* test: remove unnessary sleep in test

Signed-off-by: ZhuZiyi <[email protected]>

* fix: resolve some conflicts

Signed-off-by: ZhuZiyi <[email protected]>

* fix: resolve conversation

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* chore: code format

Signed-off-by: ZhuZiyi <[email protected]>

* feat: modify the function of schedule to synchronize and drop sender after stopping scheduler

Signed-off-by: ZhuZiyi <[email protected]>

---------

Signed-off-by: ZhuZiyi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Category Enhancements
Projects
Archived in project
2 participants