Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

raft,tracker: track follower commit index #132

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

pav-kv
Copy link
Contributor

@pav-kv pav-kv commented Jan 24, 2024

This PR closes a gap in commit index tracking, and reduces the number of unnecessary MsgApp messages in the system.

Previously, the leader did not precisely know what commit index the follower is at, and always had to send an empty MsgApp to bring it up to date if it's not.

With this change, followers now send the commit index of their logs back to the leader, and the leader tracks each follower's commit index. This allows the leader to send an empty MsgApp with a commit index update only if the tracked index is behind.

Resolves #131

@pav-kv pav-kv force-pushed the track-commit-index branch 7 times, most recently from 196927d to e85d630 Compare January 24, 2024 15:51
@pav-kv pav-kv changed the title raft: send commit index eagerly and only if necessary raft: send commit index only if necessary Jan 24, 2024
@pav-kv pav-kv force-pushed the track-commit-index branch 2 times, most recently from bbedbac to 8e5b410 Compare January 25, 2024 13:40
@pav-kv pav-kv force-pushed the track-commit-index branch 4 times, most recently from 7c2e0fc to 3723231 Compare March 4, 2024 15:18
@pav-kv pav-kv changed the title raft: send commit index only if necessary raft,tracker: track follower commit index Mar 4, 2024
When we send a snapshot, this is equivalent to sending all entries up to
the snapshot's index. Correspondingly, we update the Next index to
reflect this in-flight state.

Signed-off-by: Pavel Kalinnikov <[email protected]>
This commit adds a Progress.pendingCommit field tracking the highest
commit index <= Next-1 which the leader sent to the follower. It is used
to distinguish cases when a commit index update needs or doesn't need to
be sent to a follower.

Signed-off-by: Pavel Kalinnikov <[email protected]>
This commit closes a gap in commit index tracking.

Previously, the leader did not precisely know what commit index the
follower is at, and always had to send an empty MsgApp to brind it up to
date if it's not.

With this commit, followers now send the commit index of their logs back
to the leader, and the leader tracks each follower's commit index. This
will allow the leader (see other commits) to send an empty MsgApp with a
commit index update only if the tracked index is behind, which will
reduce the number of unnecessary messages in the system.

Signed-off-by: Pavel Kalinnikov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove redundant MsgApp sends
2 participants