Skip to content

Commit

Permalink
GITBOOK-585: update and polish a bit sync modes
Browse files Browse the repository at this point in the history
  • Loading branch information
marcindsobczak authored and gitbook-bot committed Jun 23, 2023
1 parent a023430 commit 38c2eae
Show file tree
Hide file tree
Showing 5 changed files with 41 additions and 54 deletions.
48 changes: 19 additions & 29 deletions ethereum-client/sync-modes.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,31 +3,31 @@
## There are three main synchronization modes

* snap sync
* fastest way to sync to a network (Syncs to Mainnet in \~3 hours)
* the fastest way to sync to the network (syncs to mainnet in \~3 hours)
* downloads only the latest state, headers, and optionally bodies and receipts
* fast sync
* slower than snap sync
* useful on nethermind-only chains (like Gnosis), where snap sync is not available for now
* downloads only the latest state, headers, and optionally bodies and receipts
* you can run it like this: `./Nethermind.Runner --config mainnet`
* if you have a good connection and a reasonable machine then setting --Network.MaxActivePeers to 256 (see NetworkConfig section in configuration) should give much better fast sync times (we use 256 peers to get 5 hours syncs)\
[https://docs.nethermind.io/nethermind/ethereum-client/configuration/network](https://docs.nethermind.io/nethermind/ethereum-client/configuration/network)
* archive sync
* heavy historical sync verifying all the transactions and keeping all the historical state
* you can run it like this `./Nethermind.Runner --config mainnet_archive`

| Sync Mode | Disk Space needed | Full current state | Full current and all historical states | Can sync a full archive node from this | Time to sync | Time to RPC |
| ----------------------------------------------------- | ----------------- | ------------------ | -------------------------------------- | -------------------------------------- | ------------ | ----------- |
| archive | \~12TB | YES | YES | YES | \~3 weeks | \~3 weeks |
| fast sync with all bodies and receipts | \~800GB | YES | NO | YES | \~20 hours | \~20 hours |
| default fast sync (with barriers set to support Eth2) | \~500GB | YES | NO | NO | \~11 hours | \~11 hours |
| fast sync without receipts | \~450GB | YES | NO | YES | \~12 hours | \~12 hours |
| fast sync without bodies and receipts | \~200GB | YES | NO | NO | \~9 hours | \~9 hours |
| Sync Mode | Disk Space needed | Full current state | Full current and all historical states | Can sync a full archive node from this | Time to sync | Time to RPC |
| ---------------------------------------------------------- | ----------------- | ------------------ | -------------------------------------- | -------------------------------------- | ------------ | ----------- |
| archive | \~12TB | YES | YES | YES | \~3 weeks | \~3 weeks |
| snap/fast sync with all bodies and receipts | \~800GB | YES | NO | YES | \~20 hours | \~20 hours |
| default snap/fast sync (with barriers set to support Eth2) | \~500GB | YES | NO | NO | \~11 hours | \~11 hours |
| snap/fast sync without receipts | \~450GB | YES | NO | YES | \~12 hours | \~12 hours |
| snap/fast sync without bodies and receipts | \~200GB | YES | NO | NO | \~9 hours | \~9 hours |

## Snap Sync

The Nethermind client includes beta Snap Sync support from version [v1.13.0](https://github.com/NethermindEth/nethermind/releases/tag/1.13.0). This allows a node to perform the initial synchronization and download of Ethereum’s state up to 10 times faster than before.
Snap sync allows a node to perform the initial synchronization and download of Ethereum’s state up to 10 times faster than using fast sync.

#### How to Enable

To enable make sure `SnapSync` is set to true in the Sync module of your `.cfg` file
Snap sync is enabled by default for majority of networks. To make sure, check if`SnapSync` is set to `true` in the Sync module of your `.cfg` file

```
"Sync": {
Expand All @@ -49,11 +49,9 @@ To enable make sure `SnapSync` is set to true in the Sync module of your `.cfg`
> }
> ```
#### Snap Sync VS Other Sync Modes

More than 10TB of storage is needed today to run a full archive node — one that stores all the state since genesis. This makes it so that setting up a node can take days if not weeks.
#### Snap Sync vs Other Sync Modes

An operator can instead choose to run Fast Sync when setting up the node, which downloads the state associated with the last 64 blocks that currently comes at about 90Gb. This can still take more than 24 hours in a fast machine. With Snap Sync the sync time is reduced to 2-3h with a download of 30Gb.
More than 12TB of storage is needed today to run a full archive node — one that stores all the state since genesis. Because of that, setting up an archive node can take days or even weeks. Fast Sync can still take more than 24 hours on the fast machine and download about 90GB state data. With Snap Sync, sync time is reduced to 2-3h with a download of about 30GB.

This reduction in sync time and download size has to do with the specific way in which Ethereum’s state is stored in a node: Merkle trees.

Expand All @@ -63,23 +61,15 @@ With Fast Sync, a node downloads the headers of each block and retrieves all the

#### Current limitations and future development

As of [v1.13.0](https://github.com/NethermindEth/nethermind/releases/tag/1.13.0), Snap Sync on the Nethermind client can only download the Ethereum state but not serve it to other clients implementing Snap Sync.

Since the only Ethereum client that supports serving Snap Sync requests is Geth, only networks supported by Geth can be synced: Mainnet, Goerli, Ropsten and Rinkeby.
For now Snap Sync on the Nethermind client can only download the Ethereum state but not serve it to other clients - snap serving development is in progress, expected late 2023/early 2024.

Also, as stated above, it is not advised to enable Snap Sync on already synced node as it can trigger syncing again. This will be fixed in the next update.
The only Ethereum client that supports serving Snap Sync requests is Geth, so only networks supported by Geth can be synced using that method: Mainnet, Goerli, Sepolia.

## Fast Sync

Fast sync is probably the most popular method of syncing. After completing the fast sync your node will have the ability to answer questions like _'what is my account balance **now**'_, _'how many XYZ tokens SomeExchange holds **at the moment**'_.

Fast sync is the default syncing method in Nethermind so whenever you launch standard configs you will end up using Fast Sync.

`./Nethermind.Runner --config mainnet`

`./Nethermind.Runner --config goerli`
After completing the fast sync your node will have the ability to answer questions like _'what is my account balance **now**'_, _'how many XYZ tokens SomeExchange holds **at the moment**'_.

Fast sync has multiple stages. Nethermind uses a `pivot block` number to improve fast sync performance. The `pivot block` data is provided in the configuration file and consists of the `block number`, `block hash` and `block total difficulty` (please note that `total difficulty` is different than `difficulty`). Please note that the meaning of `pivot block` is different in Nethermind than in other clients, for example. Before synchronizing state data Nethermind synchronizes in two directions - backwards from `pivot block` to 0 for headers and forward to the head of the chain for headers, blocks and receipts. Forward sync might be very slow (5 - 50 bps) so it is important that you use the latest config or update your config before synchronization.
Fast sync has multiple stages. Nethermind uses a `pivot block` number to improve fast sync performance. The `pivot block` data is automatically updated after initialization of the client and consists of the `block number`, `block hash` and `block total difficulty` (please note that `total difficulty` is different than `difficulty`). Before synchronizing state data Nethermind synchronizes in two directions - backwards from `pivot block` to 0 for headers and forward to the head of the chain for headers, blocks and receipts. Forward sync might be very slow (5 - 50 bps) so having fresh pivot block is very important - it is guaranteed by recently implemented auto-update.

After downloading the block data Nethermind will start state sync (downloading the latest state trie nodes). We are providing an estimate for the download size and progress but the real value may be different than the estimate (especially if you are using an old version of Nethermind as we sometimes manually adjust the estimator based on our observations of the chain growth rate). Because of this sometimes your sync may continue even when it shows \~100% finished. The other important component is the speed of your sync - if your IO / network / file system causes the state sync to go much slower than around 1.5MB per second on average then you will start downloading some parts of the trie over and over again. In such cases you may be surprised to see something like 58000MB / 53000MB (100%). It means that you downloaded around 5GB of data that is no longer needed. If your sync is very slow (extended beyond two days) then very likely your setup cannot catch up with the chain progress.

Expand Down
2 changes: 1 addition & 1 deletion for-developers/building-nethermind.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Installing Dependencies

To build Nethermind on any platform you will need to install **Git** and the **.NET SDK 6.0.**
To build Nethermind on any platform you will need to install **Git** and the **.NET SDK 7.0.**

{% embed url="https://git-scm.com/downloads" %}

Expand Down
27 changes: 12 additions & 15 deletions for-developers/coding-style.md
Original file line number Diff line number Diff line change
@@ -1,38 +1,35 @@
---
description: 'Nethermind C# Coding Style'
description: Nethermind C# Coding Style
---

# Coding Style

## General Rules:

* We use four spaces for indentation \(not tabs\).
* We use four spaces for indentation (not tabs).
* Do not use Hungarian Notation!
* Avoid more than one empty line at any time. For example, do not have two blank lines between members of a type.
* If a file happens to differ in style from these guidelines \(e.g. private members are named m\_member rather than \_member\), the existing style in that file takes precedence.
* We only use `var` when it's obvious what the variable type is \(e.g. `var stream = new FileStream(...)` not `var stream = OpenStandardInput()`\).
* If a file happens to differ in style from these guidelines (e.g. private members are named m\_member rather than \_member), the existing style in that file takes precedence.
* We only use `var` when it's obvious what the variable type is (e.g. `var stream = new FileStream(...)` not `var stream = OpenStandardInput()`).
* Fields should be specified at the top within type declarations.
* When using a single-statement `if`, we follow these conventions:
* Using braces is always accepted, and required if any block of an if/else if/.../else compound statement uses braces or if a single statement body spans multiple lines.
* Braces may be omitted only if the body of every block associated with an if/else if/.../else compound statement is placed on a single line.
* Braces may be omitted only if the body of every block associated with an if/else if/.../else compound statement is placed on a single line. 
* Avoid complex conditions in ternary operators.
* Avoid declaring methods with more than 5 parameters. Consider refactoring this code.
* Try to replace large parameter-sets \(> than 5 parameters\) with one or more `class` or `struct` parameters –
* Try to replace large parameter-sets (> than 5 parameters) with one or more `class` or `struct` parameters –

especially when used in multiple method signatures.
especially when used in multiple method signatures. 
* Always call `Close()` or `Dispose()` on classes that offer it. 
* Wrap instantiation of `IDisposable` objects with a “`using`” statement to ensure that Dispose() is

* Always call `Close()` or `Dispose()` on classes that offer it.
* Wrap instantiation of `IDisposable` objects with a “`using`” statement to ensure that Dispose\(\) is

automatically called.

* Always prefer aggregation over inheritance.
automatically called.
* Always prefer aggregation over inheritance. 

## Naming:

* We use `camelCase` for local variables and parameters.
* We use `_camelCase` for internal and private fields and use readonly where possible. Prefix internal and private instance fields with `_`, and thread static fields with `t_`.
* We use `snake_case` for test method names.
* We use `IPascalCase` for interfaces names \(see the `I` prefix\).
* We use `IPascalCase` for interfaces names (see the `I` prefix).
* For the rest of identifiers not specified here, we use `PascalCase`.

13 changes: 7 additions & 6 deletions for-developers/custom-analytic-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,17 @@

Check plugins for some easy addition of analytical tools

{% page-ref page="../ethereum-client/plugins.md" %}
{% content-ref url="../ethereum-client/plugins.md" %}
[plugins.md](../ethereum-client/plugins.md)
{% endcontent-ref %}

You can also read more about some useful interfaces below:

There are multiple extension points where you can add custom analytics to your Nethermind node if you know some C\#. Below you will find an example of using two very useful interfaces - `IBlockVisitor` and `ITreeVisitor`.
There are multiple extension points where you can add custom analytics to your Nethermind node if you know some C#. Below you will find an example of using two very useful interfaces - `IBlockVisitor` and `ITreeVisitor`.

Just to execute the code I have added one new initialization step that invokes two custom verifiers that I have used for calculating total supply in two different ways - by calculating mining rewards and by summing up all account balances:

```text
```
[RunnerStepDependencies(typeof(ReviewBlockTree))]
public class RunCustomTools : IStep
{
Expand Down Expand Up @@ -55,7 +57,7 @@ Just to execute the code I have added one new initialization step that invokes t

Below you will see an example of using ITreeVisitor that allows to check all the blocks, including some of the discarded branches if you wish so:

```text
```
public class RewardsVerifier : IBlockTreeVisitor
{
private ILogger _logger;
Expand Down Expand Up @@ -109,7 +111,7 @@ Below you will see an example of using ITreeVisitor that allows to check all the

And here you will find an example of a tree visitor that sums up all the account balances:

```text
```
public class SupplyVerifier : ITreeVisitor
{
private readonly ILogger _logger;
Expand Down Expand Up @@ -149,4 +151,3 @@ And here you will find an example of a tree visitor that sums up all the account
public void VisitCode(Keccak codeHash, TrieVisitContext trieVisitContext) { }
}
```

5 changes: 2 additions & 3 deletions for-developers/web3.py.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Web3.py

It is possible to connect to the Nethermind node using **web3.py** \(python web3.js implementation\). You will need to have `web3.py` installed using following guides:
It is possible to connect to the Nethermind node using **web3.py** (python web3.js implementation). You will need to have `web3.py` installed using following guides:

* [https://github.com/ethereum/web3.py](https://github.com/ethereum/web3.py)
* [https://web3py.readthedocs.io/en/stable/quickstart.html](https://web3py.readthedocs.io/en/stable/quickstart.html)
Expand All @@ -22,10 +22,9 @@ else:
print(client)
```

You should see the following output \(depends on the node version\):
You should see the following output (depends on the node version):

```bash
True
Nethermind/v1.4.8-13-5c66dcdf6-20200120/X64-Linux 5.3.2-050302-generic/Core3.1.1
```

0 comments on commit 38c2eae

Please sign in to comment.