Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync2: implement multi-peer synchronization #6358

Open
wants to merge 11 commits into
base: develop
Choose a base branch
from
Open

Commits on Oct 23, 2024

  1. sync2: improve rangesync Receive semantics

    Given that after recent item sync is done (if it's needed at all), the
    range set reconciliation algorithm no longer depends on newly received
    item being added to the set, we can save memory by not adding the
    received items during reconciliation.
    
    During real sync, the received items will be sent to the respective
    handlers and after the corresponding data are fetched and validated,
    they will be added to the database, without the need to add them to
    cloned OrderedSets which are used to sync against particular peers.
    ivan4th committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    86b9591 View commit details
    Browse the repository at this point in the history
  2. sync2: implement multi-peer synchronization

    This adds multi-peer synchronization support.
    When the local set differs too much from the remote sets,
    "torrent-style" "split sync" is attempted which splits the set into
    subranges and syncs each sub-range against a separate peer.
    Otherwise, the full sync is done, syncing the whole set against
    each of the synchronization peers.
    Full sync is also done after each split sync run.
    The local set can be considered synchronized after the specified
    number of full syncs has happened.
    
    The approach is loosely based on [SREP: Out-Of-Band Sync of
    Transaction Pools for Large-Scale
    Blockchains](https://people.bu.edu/staro/2023-ICBC-Novak.pdf) paper by
    Novak Boškov, Sevval Simsek, Ari Trachtenberg, and David Starobinski.
    ivan4th committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    ef30f47 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2024

  1. Configuration menu
    Copy the full SHA
    af95c7f View commit details
    Browse the repository at this point in the history
  2. sync2: doc update

    ivan4th committed Oct 26, 2024
    Configuration menu
    Copy the full SHA
    b062eaa View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2024

  1. Configuration menu
    Copy the full SHA
    3bfaa3f View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2024

  1. Configuration menu
    Copy the full SHA
    ef484c5 View commit details
    Browse the repository at this point in the history
  2. sync2: address comments

    ivan4th committed Oct 30, 2024
    Configuration menu
    Copy the full SHA
    0cf1678 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    287d76d View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    51166fa View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2024

  1. Configuration menu
    Copy the full SHA
    c58d690 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    cf46587 View commit details
    Browse the repository at this point in the history