Sync state subscription #244
Open
+652
−169
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
biryukovmaxim
force-pushed
the
sync_state_subscription
branch
2 times, most recently
from
ad9bd5c
to
e05e3b6
Compare
biryukovmaxim
force-pushed
the
sync_state_subscription
branch
from
7360a42
to
777748c
Compare
biryukovmaxim
force-pushed
the
sync_state_subscription
branch
2 times, most recently
from
b321a53
to
d339cd5
Compare
biryukovmaxim
force-pushed
the
sync_state_subscription
branch
3 times, most recently
from
71cd277
to
8acba81
Compare
This update adds notifications for sync state changes in the consensus process. These changes were necessary to keep track of the block syncing progress. `ConsensusServices` was modified to include the notification root, and the `syncStateChangedRequest` and `syncStateChangedNotification` RPC calls were added to allow remote retrieval of sync state. Minor syntactical improvements were also made to adhere to best coding practices.
This commit introduces the ability to report the progress of Initial Block Download (IBD) to the notification system. This feature was added to the `ProgressReporter` struct in the `progress.rs` file, where a callback function was incorporated to handle the reporting. The callback function is invoked during batch processing and completion reporting in the IBD. Furthermore, new changes were made for the `SyncStateChangedNotification`, adding a variant for header progress reporting. Consequently, this needed adjustment also in proto files for proper message handling. This improves the communication and interaction with the wallet; hence enhancing user experience by keeping them updated on the IBD progress.
The extra sync state, "Blocks", has been added for the purpose of block synchronization. The changes include adding the fields `blocks` and `progress` to the `BlocksState` message in the `rpc.proto` file, as well as corresponding implementations across the codebase. This allows the synchronization state to be accurately monitored and predicted, which can help improve the performance and stability of the system. These changes are expected to improve the system's understanding of state changes during block synchronization and alter respective notifications.
Implemented the UTXO resync notification feature in the UTXO Index, notifying the user when the UTXO index starts resyncing. This is done by adding a new UtxoResyncState in the SyncStateChangedNotification enum and notifying the user when the resync event occurs. Dependencies were updated in Cargo.toml and the notification was integrated into the GRPC layer.
Implemented UtxoSync mechanism in the SyncStateChangedNotification. It is designed to provide more detailed information about sync state by additionally conveying the number of chunks and total progress in its structure. This enhances the UTXO synchronization process by allowing users to track the status of synced chunks, significantly improving process visibility and debuggability.
Added a new state called TrustSync to the SyncStateChangedNotification enum to share information about the processing of trusted blocks. The addition of TrustSync allows the program to notify about the number of trusted blocks processed in the last one second and the total number of such blocks processed till now. This is essential for improving the supervision of the synchronization process.
This commit removes a redundant `take().unwrap()` of the `notification_root` field and the subsequent reassignment of it. Now the field is directly accessed via `as_ref().unwrap()`. This change removes unnecessary operations, and might slightly improve performance.
Added a new "Synced" state to the `SyncStateChangedNotification` enum, which gives more precise control over sync state notifications. This was reflected across RPC and consensus notify modules, including additions to GRPC proto definitions and corresponding conversion functions. This change enables better tracking and handling of the sync completed state.
Modified the consensus parameters to group Difficulty Adjustment Algorithm (DAA) parameters into their dedicated struct. The changes affect how these parameters are retrieved and set across the mining, testing and consensus codebase. Structured DAA parameters within their own entity enhances code readability and manageability.
remove the use of set method to assign consensus_manager in integration_tests.rs
The SYNC_STATE import was removed from the integration_tests.rs file because it was unused. This was probably a leftover from some previously refactored code, and its presence might have led to confusion, hence it has been removed.
Added SyncStateChanged handling in various files including kaspad.rs, message.rs and notification.rs in the GRPC core, which allows the system to notify and handle changes in the sync state.
The commit includes `SyncStateChangedNotification` into `RpcApiOps` array in the `client.rs` file. This will allow further operations to incorporate the latest sync state changes effectively.
This change corrects the function call made for the SerdeJson encoding in the connection.rs file. Previously, it was mistakenly referencing the Borsh encoding function instead of the appropriate SerdeJson one.
A typo in the regex pattern for processing block headers during Initial Block Download(IBD) resulted in incorrect matches. This commit fixes the regex by removing the extra square brackets.
The sync_state.rs file in the consensus/src directory has been removed as it is no longer needed. All references to SYNC_STATE which was previously declared in this file have been replaced with suitable alternatives. This was done to streamline the consensus and avoid unnecessary complexities.
Refactored the sync condition calculation in both pruning_proof/mod.rs and virtual_processor/processor.rs by moving the logic to the daa_window_params.is_nearly_synced method. This eliminates redundant code and enhances readability
The sink processing was streamlined to eliminate an unnecessary step. The compact header data is now directly retrieved from the headers_store using a new_sink variable. This change simplifies the codebase and may potentially enhance the performance of headers_store data retrieval.
Improve handling of 'is_nearly_synced' state in processor.rs, adding a guard not to send 'SyncStateChangedNotification' continuously. Now 'SyncStateChangedNotification' is sent only when the state changes. The state 'is_nearly_synced' is now calculated outside of the if block to handle cases when getting 'ghostdag_data.selected_parent' might fail. 'previous_synced' flag was added to 'VirtualStateProcessor' structure to store previous state and send notification only when the state changes. This reduces redundant notifications, improving the performance. In 'pruning_proof', we now use a safe get by 'is_ok_and' method and inside use a lambda to calculate 'is_synced'. This change makes it easier to understand the code and handle the state correctly.
biryukovmaxim
force-pushed
the
sync_state_subscription
branch
from
8acba81
to
1961717
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.