From 95a3630d0a7b859055c26130d345e0a9e7696896 Mon Sep 17 00:00:00 2001 From: Ruben Buniatyan Date: Sat, 2 Sep 2023 22:13:11 +0200 Subject: [PATCH] Reorg fundamentals --- docs/07-resources/faq.md | 6 +- docs/fundamentals/06-logs/_category_.json | 11 --- .../06-logs/logging-configuration.md | 54 --------------- docs/fundamentals/07-database.md | 31 --------- .../_category_.json | 8 --- .../resync-database-from-scratch.md | 67 ------------------- .../09-private-networks/_category_.json | 11 --- docs/fundamentals/configuration.md | 2 +- .../README.md => database.md} | 33 ++++++++- docs/fundamentals/{06-logs => }/logs.md | 62 +++++++++++++++-- .../private-networks/_category_.json | 9 +++ .../clique.md} | 5 +- .../overview.md} | 6 +- .../spaceneth.md} | 5 +- .../full-pruning.md => pruning.md} | 9 +-- docs/fundamentals/running-nethermind.md | 2 +- .../{05-security.md => security.md} | 5 +- .../{03-sync-modes.md => sync.md} | 67 ++++++++++++++++++- docs/get-started/system-requirements.md | 4 +- 19 files changed, 184 insertions(+), 213 deletions(-) delete mode 100644 docs/fundamentals/06-logs/_category_.json delete mode 100644 docs/fundamentals/06-logs/logging-configuration.md delete mode 100644 docs/fundamentals/07-database.md delete mode 100644 docs/fundamentals/08-how-to-reduce-database-size/_category_.json delete mode 100644 docs/fundamentals/08-how-to-reduce-database-size/resync-database-from-scratch.md delete mode 100644 docs/fundamentals/09-private-networks/_category_.json rename docs/fundamentals/{08-how-to-reduce-database-size/README.md => database.md} (53%) rename docs/fundamentals/{06-logs => }/logs.md (57%) create mode 100644 docs/fundamentals/private-networks/_category_.json rename docs/fundamentals/{09-private-networks/how-to-setup-a-nethermind-only-clique-based-chain.md => private-networks/clique.md} (98%) rename docs/fundamentals/{09-private-networks/README.md => private-networks/overview.md} (97%) rename docs/fundamentals/{09-private-networks/how-to-setup-a-nethermind-only-spaceneth-based-chain.md => private-networks/spaceneth.md} (99%) rename docs/fundamentals/{08-how-to-reduce-database-size/full-pruning.md => pruning.md} (98%) rename docs/fundamentals/{05-security.md => security.md} (86%) rename docs/fundamentals/{03-sync-modes.md => sync.md} (76%) diff --git a/docs/07-resources/faq.md b/docs/07-resources/faq.md index fd53f01ed..5828076b5 100644 --- a/docs/07-resources/faq.md +++ b/docs/07-resources/faq.md @@ -55,17 +55,17 @@ If you have any issues, please reach out to us on Discord: [https://discord.gg/X ## Can I disable logging to file? -You can find more details on the logging config page [here](../fundamentals/06-logs/logging-configuration.md) +You can find more details on the logging config page [here](../fundamentals/logs.md) ## Can I disable logging of JSON RPC calls? -You can find more details on the logging config page [here](../fundamentals/06-logs/logging-configuration.md) +You can find more details on the logging config page [here](../fundamentals/logs.md) ## How can I configure validator on AuRa / Clique? You can find more details on running validators in the docs -> for [AuRa](./guides-and-helpers/validator-setup/aura-validator.md) and -for [Clique](../fundamentals/09-private-networks/how-to-setup-a-nethermind-only-clique-based-chain.md). +for [Clique](../fundamentals/private-networks/clique.md). You can learn more about how to setup a aura validator [here](./guides-and-helpers/validator-setup/aura-validator.md) diff --git a/docs/fundamentals/06-logs/_category_.json b/docs/fundamentals/06-logs/_category_.json deleted file mode 100644 index e394f221f..000000000 --- a/docs/fundamentals/06-logs/_category_.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "position": 6, - "label": "Logs", - "collapsible": true, - "link": { - "type": "generated-index" - }, - "customProps": { - "description": "General overview and introduction of Nethermind Execution Client" - } -} diff --git a/docs/fundamentals/06-logs/logging-configuration.md b/docs/fundamentals/06-logs/logging-configuration.md deleted file mode 100644 index 7ab5d82b5..000000000 --- a/docs/fundamentals/06-logs/logging-configuration.md +++ /dev/null @@ -1,54 +0,0 @@ -# Logging Configuration - -## Log config file location - -Logging in Nethermind is done via NLog library that can be configured by editing the NLog.config file. - -| Environment Type | NLog.config location | -|-----------------------------------------------------------------------------------|------------------------------------------------------------------------| -| built from src - Debug mode | src\Nethermind\Nethermind.Runner\bin\Debug\netcoreapp3.1\NLog.config | -| built from src - Release mode | src\Nethermind\Nethermind.Runner\bin\Release\netcoreapp3.1\NLog.config | -| PPA | /usr/share/nethermind/NLog.config | -| Docker | /nethermind/NLog.config | -| from [downloads page](https://downloads.nethermind.io) | top level directory after unzipping the package | -| from [GitHub releases page](https://github.com/NethermindEth/nethermind/releases) | top level directory after unzipping the package | -| dAppNode | ? \[to be documented] | - -## Log config file syntax - -Detailed NLog configuration options can be found -here: [https://nlog-project.org/config/](https://nlog-project.org/config/) - -## Config or CLI log rules - -Simple logging rules can be added through configuration file or command line argument. - -For example this would add `Trace` level logs to any logger under `Synchronization` module and `Debug` level logs -for `BlockTree` from `Blockchain` module:\ -`--Init.LogRules Synchronization.*:Trace;Blockchain.BlockTree:Debug` - -## Global logging override - -Additionally there are global logging override that you can use temporarily: - -| Command line override | Log level | -|--------------------------------------------------|-----------| -| ./Nethermind.Runner --config mainnet --log TRACE | TRACE | -| ./Nethermind.Runner --config mainnet --log DEBUG | DEBUG | -| ./Nethermind.Runner --config mainnet --log INFO | INFO | -| ./Nethermind.Runner --config mainnet --log WARN | WARN | -| ./Nethermind.Runner --config mainnet --log ERROR | ERROR | - -## JSON RPC logging level - -This can be done by including these lines in the logging configuration file: - -``` - - - -``` - -## Enterprise Logging - -See how to configure Seq [here](https://docs.nethermind.io/nethermind/enterprise/seq) diff --git a/docs/fundamentals/07-database.md b/docs/fundamentals/07-database.md deleted file mode 100644 index b34234c6f..000000000 --- a/docs/fundamentals/07-database.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: dbpath and databases info ---- - -# Database - -Nethermind uses the [RocksDB](https://rocksdb.org) database to store state. By default the database is stored in the -same directory where the client binaries are. You can change it by providing a `--baseDbPath` config switch in the -command line, e.g.`./Nethermind.Runner --config goerli --baseDbPath /home/username/nethermind_db` - -After Nethermind is started, you will see multiple directories appearing in the _baseDbPath_ directory. - -![Example of the DB directory on a freshly deployed Ubuntu VM with Nethermind.]() - -| DB Directory | Content | -|-------------------|------------------------------------------------------------------------------------------------------------------------------------------| -| blockInfos | information about blocks at each level of the block tree (canonical chain and branches) | -| blocks | block bodies (block transactions and uncle data) | -| bloom | bloom indexes for fast log searches | -| canonicalHashTrie | LES protocol related data | -| code | contract bytecodes | -| discoveryNodes | peers discovered via discovery protocol - used for quick peering after restarts (you can copy this DB between nodes to speed up peering) | -| headers | block headers only | -| pendingTx | (this DB is wiped out on each restart) 2nd level cache of pending transactions / mempool (1st level is in memory) | -| peers | stores additional sync peers information (like peer reputation) - you can copy this DB between nodes to speed up peering on fresh sync | -| receipts | transaction receipts data | -| state | blockchain state including accounts and contract storage (Patricia trie nodes) | - -You can use `rsync` between your nodes to clone the database (One of our users copied entire 4.5TB archive state this -way while the node was running and only stopped the node for the very last stage of `rsync` ). You can also simply copy -the database between Unix and Windows systems (and most likely macOS). diff --git a/docs/fundamentals/08-how-to-reduce-database-size/_category_.json b/docs/fundamentals/08-how-to-reduce-database-size/_category_.json deleted file mode 100644 index 5d77e2aad..000000000 --- a/docs/fundamentals/08-how-to-reduce-database-size/_category_.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "position": 8, - "label": "How to reduce the size of the database", - "collapsible": true, - "link": { - "type": "generated-index" - } -} diff --git a/docs/fundamentals/08-how-to-reduce-database-size/resync-database-from-scratch.md b/docs/fundamentals/08-how-to-reduce-database-size/resync-database-from-scratch.md deleted file mode 100644 index ab238fd4e..000000000 --- a/docs/fundamentals/08-how-to-reduce-database-size/resync-database-from-scratch.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -description: How to resync your Nethermind node ---- - -# Resync database from scratch - -## Resyncing a Nethermind Node - -This guide explains how to resync a Nethermind node using the existing Pivot block or updating it to a more recent one. - -### Steps to Resync a Nethermind Node - -1. **Stop the Nethermind node**: If your Nethermind node is currently running, stop it to ensure that no new data is - being written to the database during the resync process. -2. **Delete the existing database**: Navigate to the Nethermind data directory. The location of this directory depends - on how Nethermind was installed and your configuration settings. Inside the data directory, find the `nethermind_db` - folder and delete the `mainnet` subfolder to remove the existing database for the mainnet. -3. **Update the configuration file (optional)**: If you want to change any configuration settings before resyncing the - node, edit the `mainnet.cfg` file located in the Nethermind directory. For example, you might want to adjust the - pruning settings or specify a different network. -4. **Update the Pivot block (optional)**\ - :::danger - **Only for versions before 1.19.0 where Auto-Pivot approach was introduced** - ::: - 1. **Using Etherscan**: If you want to speed up the syncing process, you can update the Pivot block to a more recent - one. To do this, find the `Sync` section in the `mainnet.cfg` file and update the `PivotNumber` and `PivotHash` - fields to match a recent "finalized" block number and its corresponding hash. You can obtain this information from - a block explorer such as [Etherscan](https://etherscan.io/).\ - \ - Using block number 17165278 from [Etherscan](https://etherscan.io/block/17165278): - - ``` - { - "PivotNumber": 17165278, - "PivotHash": "0xa665315efd923f3b11215feee09a9d3e13c5e6ee602fa19b642824682ec0a752" - } - ``` - 2. **Using Nethermind's GitHub**: Alternatively, you can update the Pivot block by referring to - the [Nethermind's mainnet.cfg file on GitHub](https://github.com/NethermindEth/nethermind/blob/master/src/Nethermind/Nethermind.Runner/configs/mainnet.cfg). - The Pivot block is periodically bumped to the HEAD-8192 block of the mainnet chain. Copy the `PivotNumber` - and `PivotHash` values from the GitHub file and update your local `mainnet.cfg` file accordingly. -5. **Restart the Nethermind node**: Start the Nethermind node again to initiate the resync process. The node will begin - syncing from the existing Pivot block or the specified updated Pivot block, downloading and processing all the blocks - in the blockchain. - -To ensure that your Nethermind node is resyncing, you can monitor the logs for the node's progress. The logs will -display information about the block processing, synchronization status and `OldHeaders` being processed. By observing -the increasing block numbers and synchronization messages in the logs, you can confirm that the resync process is active -and working as expected. - -> Old Headers 0 / 17154000 | queue 0 | current 0.00bps | total 0.00bps -> -> Old Headers 768 / 17154000 | queue 0 | current 766.07bps | total 762.49bps -> -> Beacon Headers from block 17154001 to block 17169722 | 960 / 15722 | queue 4992 | current 0.00bps | total -> 40622848.83bps -> -> Old Headers 9024 / 17154000 | queue 0 | current 576.40bps | total 1286.40bps -> -> Beacon Headers from block 17154001 to block 17169723 | 9024 / 15723 | queue 6698 | current 2694.81bps | total -> 3882943.63bps\ -> Downloaded 17154031 / 17172359 | current 0.00bps | total 0.00bps -> -> Downloaded 17154062 / 17169724 | current 0.00bps | total 2.88bps - -Keep in mind that resyncing a Nethermind node can take a considerable amount of time. It depends on your hardware, -internet connection, and the size of the blockchain. diff --git a/docs/fundamentals/09-private-networks/_category_.json b/docs/fundamentals/09-private-networks/_category_.json deleted file mode 100644 index 646f0a393..000000000 --- a/docs/fundamentals/09-private-networks/_category_.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "position": 2, - "label": "Private networks", - "collapsible": true, - "link": { - "type": "generated-index" - }, - "customProps": { - "description": "General overview and introduction of Nethermind Execution Client" - } -} diff --git a/docs/fundamentals/configuration.md b/docs/fundamentals/configuration.md index a2420e81e..cb37721e7 100644 --- a/docs/fundamentals/configuration.md +++ b/docs/fundamentals/configuration.md @@ -72,7 +72,7 @@ The command line options are case-sensitive and can be defined only once unless

-- **`-c, --configsDirectory `** +- **`-cd, --configsDirectory `** An absolute or relative path to the configuration files directory. Defaults to `configs`. diff --git a/docs/fundamentals/08-how-to-reduce-database-size/README.md b/docs/fundamentals/database.md similarity index 53% rename from docs/fundamentals/08-how-to-reduce-database-size/README.md rename to docs/fundamentals/database.md index 27402e69f..8a844af74 100644 --- a/docs/fundamentals/08-how-to-reduce-database-size/README.md +++ b/docs/fundamentals/database.md @@ -1,8 +1,35 @@ --- -description: Description of all possible options, their pros and cons +title: Database +sidebar_position: 5 --- -# How to reduce database size +Nethermind uses the [RocksDB](https://rocksdb.org) database to store state. By default the database is stored in the +same directory where the client binaries are. You can change it by providing a `--baseDbPath` config switch in the +command line, e.g.`./Nethermind.Runner --config goerli --baseDbPath /home/username/nethermind_db` + +After Nethermind is started, you will see multiple directories appearing in the _baseDbPath_ directory. + +![Example of the DB directory on a freshly deployed Ubuntu VM with Nethermind.]() + +| DB Directory | Content | +|-------------------|------------------------------------------------------------------------------------------------------------------------------------------| +| blockInfos | information about blocks at each level of the block tree (canonical chain and branches) | +| blocks | block bodies (block transactions and uncle data) | +| bloom | bloom indexes for fast log searches | +| canonicalHashTrie | LES protocol related data | +| code | contract bytecodes | +| discoveryNodes | peers discovered via discovery protocol - used for quick peering after restarts (you can copy this DB between nodes to speed up peering) | +| headers | block headers only | +| pendingTx | (this DB is wiped out on each restart) 2nd level cache of pending transactions / mempool (1st level is in memory) | +| peers | stores additional sync peers information (like peer reputation) - you can copy this DB between nodes to speed up peering on fresh sync | +| receipts | transaction receipts data | +| state | blockchain state including accounts and contract storage (Patricia trie nodes) | + +You can use `rsync` between your nodes to clone the database (One of our users copied entire 4.5TB archive state this +way while the node was running and only stopped the node for the very last stage of `rsync` ). You can also simply copy +the database between Unix and Windows systems (and most likely macOS). + +## How to reduce database size The Nethermind database can experience substantial growth over time, starting from an initial size of approximately 650 GB. As a result, many node-setups are configured to run on 1TB disks. However, even with the application of settings @@ -11,7 +38,7 @@ designed to slow the growth rate, these disks may eventually run out of free spa Current options to reduce db size are: 1. [Re-sync database from scratch](resync-database-from-scratch.md) -2. [Full pruning](full-pruning.md) +2. [Full pruning](../pruning.md) The table below presents a short comparison of these methods including possible fine-tuning of each method. Data was fetched from a node running on a machine with the below specifications:\ diff --git a/docs/fundamentals/06-logs/logs.md b/docs/fundamentals/logs.md similarity index 57% rename from docs/fundamentals/06-logs/logs.md rename to docs/fundamentals/logs.md index 4e44bb796..c53e9ed14 100644 --- a/docs/fundamentals/06-logs/logs.md +++ b/docs/fundamentals/logs.md @@ -1,11 +1,65 @@ --- -description: A quick start example for Görli Testnet Nethermind synchronization +title: Logs +sidebar_position: 4 --- -# Explaining Nethermind logs +## Log config file location + +Logging in Nethermind is done via NLog library that can be configured by editing the NLog.config file. + +| Environment Type | NLog.config location | +|-----------------------------------------------------------------------------------|------------------------------------------------------------------------| +| built from src - Debug mode | src\Nethermind\Nethermind.Runner\bin\Debug\netcoreapp3.1\NLog.config | +| built from src - Release mode | src\Nethermind\Nethermind.Runner\bin\Release\netcoreapp3.1\NLog.config | +| PPA | /usr/share/nethermind/NLog.config | +| Docker | /nethermind/NLog.config | +| from [downloads page](https://downloads.nethermind.io) | top level directory after unzipping the package | +| from [GitHub releases page](https://github.com/NethermindEth/nethermind/releases) | top level directory after unzipping the package | +| dAppNode | ? \[to be documented] | + +## Log config file syntax + +Detailed NLog configuration options can be found +here: [https://nlog-project.org/config/](https://nlog-project.org/config/) + +## Config or CLI log rules + +Simple logging rules can be added through configuration file or command line argument. + +For example this would add `Trace` level logs to any logger under `Synchronization` module and `Debug` level logs +for `BlockTree` from `Blockchain` module:\ +`--Init.LogRules Synchronization.*:Trace;Blockchain.BlockTree:Debug` + +## Global logging override + +Additionally there are global logging override that you can use temporarily: + +| Command line override | Log level | +|--------------------------------------------------|-----------| +| ./Nethermind.Runner --config mainnet --log TRACE | TRACE | +| ./Nethermind.Runner --config mainnet --log DEBUG | DEBUG | +| ./Nethermind.Runner --config mainnet --log INFO | INFO | +| ./Nethermind.Runner --config mainnet --log WARN | WARN | +| ./Nethermind.Runner --config mainnet --log ERROR | ERROR | + +## JSON RPC logging level + +This can be done by including these lines in the logging configuration file: + +``` + + + +``` + +## Enterprise Logging + +See how to configure Seq [here](https://docs.nethermind.io/nethermind/enterprise/seq) + +## Explaining Nethermind logs You can check the supported operating systems, architectures and hardware requirements -here: [system-requirements.md](../../get-started/system-requirements.md) +here: [system-requirements.md](../get-started/system-requirements.md) After the node starts, you will see some initial info about the node and then the sync will start. Görli fast sync uses a `fast blocks` sync mode initially. The `fast blocks` sync picks some known `pivot block` from the past and @@ -85,4 +139,4 @@ Also, every now and then, a peer report will appear like below: * Receipts Transfer * Node Data Transfer * Snap Ranges Transfer -5. Fifth bracket is for Client Info like Client Name, Client Version, Operating System and Language Version. +5. Fifth bracket is for Client Info like Client Name, Client Version, Operating System and Language Version. \ No newline at end of file diff --git a/docs/fundamentals/private-networks/_category_.json b/docs/fundamentals/private-networks/_category_.json new file mode 100644 index 000000000..d16dbc48d --- /dev/null +++ b/docs/fundamentals/private-networks/_category_.json @@ -0,0 +1,9 @@ +{ + "position": 6, + "label": "Private networks", + "collapsible": true, + "link": { + "type": "doc", + "id": "overview" + } +} diff --git a/docs/fundamentals/09-private-networks/how-to-setup-a-nethermind-only-clique-based-chain.md b/docs/fundamentals/private-networks/clique.md similarity index 98% rename from docs/fundamentals/09-private-networks/how-to-setup-a-nethermind-only-clique-based-chain.md rename to docs/fundamentals/private-networks/clique.md index f07394e11..ba614468b 100644 --- a/docs/fundamentals/09-private-networks/how-to-setup-a-nethermind-only-clique-based-chain.md +++ b/docs/fundamentals/private-networks/clique.md @@ -1,9 +1,8 @@ --- -description: Set of validators sealing blocks on private clique network +title: Clique-based chain +sidebar_position: 1 --- -# How to setup a Nethermind only Clique based chain - ### TL;DR Download a script that will do all the steps described below for you. It will prompt you 2 things: diff --git a/docs/fundamentals/09-private-networks/README.md b/docs/fundamentals/private-networks/overview.md similarity index 97% rename from docs/fundamentals/09-private-networks/README.md rename to docs/fundamentals/private-networks/overview.md index cafd54d19..ebebad2c6 100644 --- a/docs/fundamentals/09-private-networks/README.md +++ b/docs/fundamentals/private-networks/overview.md @@ -1,9 +1,9 @@ --- -description: Private networks of Nethermind Client +title: Private networks +sidebar_position: 0 +slug: ./ --- -# Private Networks - It is possible to configure a private network \(private blockchain\) using Nethermind nodes. Before you setup a private network you will need to make a few decisions. One of them is choosing a consensus protocol that will be used for securing the network. diff --git a/docs/fundamentals/09-private-networks/how-to-setup-a-nethermind-only-spaceneth-based-chain.md b/docs/fundamentals/private-networks/spaceneth.md similarity index 99% rename from docs/fundamentals/09-private-networks/how-to-setup-a-nethermind-only-spaceneth-based-chain.md rename to docs/fundamentals/private-networks/spaceneth.md index 198bc01b1..7248960b9 100644 --- a/docs/fundamentals/09-private-networks/how-to-setup-a-nethermind-only-spaceneth-based-chain.md +++ b/docs/fundamentals/private-networks/spaceneth.md @@ -1,4 +1,7 @@ -# How to setup a Nethermind only Spaceneth based chain +--- +title: Spaceneth-based chain +sidebar_position: 2 +--- Spaceneth private network setup looks very similar to the above Clique setup. However, there are few major differences and will be described below. diff --git a/docs/fundamentals/08-how-to-reduce-database-size/full-pruning.md b/docs/fundamentals/pruning.md similarity index 98% rename from docs/fundamentals/08-how-to-reduce-database-size/full-pruning.md rename to docs/fundamentals/pruning.md index 8a286da29..4cc6c90a2 100644 --- a/docs/fundamentals/08-how-to-reduce-database-size/full-pruning.md +++ b/docs/fundamentals/pruning.md @@ -1,11 +1,8 @@ --- -description: >- - Explanation of Full Pruning, including its requirements and steps to execute - it. +title: Pruning +sidebar_position: 5 --- -# Full Pruning - ## Overview Full Pruning pertains to the elimination or cleanup of obsolete historical data to optimize disk space. Clients are @@ -242,4 +239,4 @@ During the Full Pruning process, several things should be kept in mind: complete successfully. 4. Several things can be done to reduce the size of the database after syncing (such as setting AncientBarriers to a proper value higher than 0, using CL which requires less disk space, and setting logs to the lowest level to avoid - log spamming). + log spamming). \ No newline at end of file diff --git a/docs/fundamentals/running-nethermind.md b/docs/fundamentals/running-nethermind.md index e947d98bc..4f0fa33e0 100644 --- a/docs/fundamentals/running-nethermind.md +++ b/docs/fundamentals/running-nethermind.md @@ -101,6 +101,6 @@ Run the launcher as follows: 1. To run a node, select `Ethereum Node` 2. Then, select the desired network (e.g., Mainnet, Sepolia, Goerli) -3. Then, select the [sync mode](./03-sync-modes.md) and you're ready to go +3. Then, select the [sync mode](sync.md) and you're ready to go If you need just the Nethermind command line interface (CLI), select `CLI` in the first step. \ No newline at end of file diff --git a/docs/fundamentals/05-security.md b/docs/fundamentals/security.md similarity index 86% rename from docs/fundamentals/05-security.md rename to docs/fundamentals/security.md index de1572158..d2764f1a9 100644 --- a/docs/fundamentals/05-security.md +++ b/docs/fundamentals/security.md @@ -1,7 +1,6 @@ --- -description: >- - Security measures that should be taken before running Nethermind Node on - Ethereum Mainnet +title: Security +sidebar_position: 3 --- # Security diff --git a/docs/fundamentals/03-sync-modes.md b/docs/fundamentals/sync.md similarity index 76% rename from docs/fundamentals/03-sync-modes.md rename to docs/fundamentals/sync.md index adbff7ca8..862e055ba 100644 --- a/docs/fundamentals/03-sync-modes.md +++ b/docs/fundamentals/sync.md @@ -1,4 +1,7 @@ -# Sync modes +--- +title: Sync +sidebar_position: 2 +--- ## There are three main synchronization modes @@ -179,3 +182,65 @@ Explanation of some data in the logs: hand when blocks are filled with very heavy transactions then `bps` might be very low while `mgasps` will be very high. It is even possible that you will see a lot of very light transactions where `tps` will be high while `bps` and `mgasps` will be average. + +## Resync a node from scratch + +This guide explains how to resync a Nethermind node using the existing Pivot block or updating it to a more recent one. + +### Steps to Resync a Nethermind Node + +1. **Stop the Nethermind node**: If your Nethermind node is currently running, stop it to ensure that no new data is + being written to the database during the resync process. +2. **Delete the existing database**: Navigate to the Nethermind data directory. The location of this directory depends + on how Nethermind was installed and your configuration settings. Inside the data directory, find the `nethermind_db` + folder and delete the `mainnet` subfolder to remove the existing database for the mainnet. +3. **Update the configuration file (optional)**: If you want to change any configuration settings before resyncing the + node, edit the `mainnet.cfg` file located in the Nethermind directory. For example, you might want to adjust the + pruning settings or specify a different network. +4. **Update the Pivot block (optional)**\ + :::danger + **Only for versions before 1.19.0 where Auto-Pivot approach was introduced** + ::: + 1. **Using Etherscan**: If you want to speed up the syncing process, you can update the Pivot block to a more recent + one. To do this, find the `Sync` section in the `mainnet.cfg` file and update the `PivotNumber` and `PivotHash` + fields to match a recent "finalized" block number and its corresponding hash. You can obtain this information from + a block explorer such as [Etherscan](https://etherscan.io/).\ + \ + Using block number 17165278 from [Etherscan](https://etherscan.io/block/17165278): + + ``` + { + "PivotNumber": 17165278, + "PivotHash": "0xa665315efd923f3b11215feee09a9d3e13c5e6ee602fa19b642824682ec0a752" + } + ``` + 2. **Using Nethermind's GitHub**: Alternatively, you can update the Pivot block by referring to + the [Nethermind's mainnet.cfg file on GitHub](https://github.com/NethermindEth/nethermind/blob/master/src/Nethermind/Nethermind.Runner/configs/mainnet.cfg). + The Pivot block is periodically bumped to the HEAD-8192 block of the mainnet chain. Copy the `PivotNumber` + and `PivotHash` values from the GitHub file and update your local `mainnet.cfg` file accordingly. +5. **Restart the Nethermind node**: Start the Nethermind node again to initiate the resync process. The node will begin + syncing from the existing Pivot block or the specified updated Pivot block, downloading and processing all the blocks + in the blockchain. + +To ensure that your Nethermind node is resyncing, you can monitor the logs for the node's progress. The logs will +display information about the block processing, synchronization status and `OldHeaders` being processed. By observing +the increasing block numbers and synchronization messages in the logs, you can confirm that the resync process is active +and working as expected. + +> Old Headers 0 / 17154000 | queue 0 | current 0.00bps | total 0.00bps +> +> Old Headers 768 / 17154000 | queue 0 | current 766.07bps | total 762.49bps +> +> Beacon Headers from block 17154001 to block 17169722 | 960 / 15722 | queue 4992 | current 0.00bps | total +> 40622848.83bps +> +> Old Headers 9024 / 17154000 | queue 0 | current 576.40bps | total 1286.40bps +> +> Beacon Headers from block 17154001 to block 17169723 | 9024 / 15723 | queue 6698 | current 2694.81bps | total +> 3882943.63bps\ +> Downloaded 17154031 / 17172359 | current 0.00bps | total 0.00bps +> +> Downloaded 17154062 / 17169724 | current 0.00bps | total 2.88bps + +Keep in mind that resyncing a Nethermind node can take a considerable amount of time. It depends on your hardware, +internet connection, and the size of the blockchain. diff --git a/docs/get-started/system-requirements.md b/docs/get-started/system-requirements.md index 80a523f4c..fd6fa514f 100644 --- a/docs/get-started/system-requirements.md +++ b/docs/get-started/system-requirements.md @@ -49,7 +49,7 @@ Nonetheless, for a comprehensive understanding of disk growth and usage, we reco #### Database growth -Nethermind requires approximately 504 GB of disk space (as of May 2023) after a fresh Ethereum Mainnet sync using default parameters. This size increases over time as the Ethereum chain grows. The node's database is in its most optimal state immediately after a sync or full pruning. Following the initial sync, the database grows at a rate of approximately 27 GB per week. To maintain this process, occasional resyncing or pruning of the node is necessary to bring it back to its optimal database size. For more info on managing node disk usage growth, see [how to reduce database size](../fundamentals/08-how-to-reduce-database-size/README.md).
+Nethermind requires approximately 504 GB of disk space (as of May 2023) after a fresh Ethereum Mainnet sync using default parameters. This size increases over time as the Ethereum chain grows. The node's database is in its most optimal state immediately after a sync or full pruning. Following the initial sync, the database grows at a rate of approximately 27 GB per week. To maintain this process, occasional resyncing or pruning of the node is necessary to bring it back to its optimal database size. For more info on managing node disk usage growth, see [how to reduce database size](../fundamentals/sync.md#resync-a-node-from-scratch).
Having a larger disk space allocation reduces the need for frequent maintenance and alleviates concerns about the chain outgrowing the available space. It's worth noting that the only drawback of running a smaller disk is the requirement for more regular resyncing or pruning. We believe that a 2 TB disk will suffice for most users. However, the choice between 1 TB and 2 TB depends on factors such as hardware costs, cloud provider expenses, and individual requirements. #### Disk speed @@ -64,7 +64,7 @@ A node can be run using different syncing options, and each option has different - **Ancient barriers.** Nethermind allows you to specify how many old block bodies and receipts you want to store. By default, Nethermind sets the ancient barrier at block 11052984. This block is significant because it marks the deployment of the deposit contract required for consensus client deposit scanning for validators. Peers and JSON-RPC requests will not have access to block bodies and receipts older than the ancient barrier. - **Non-validator mode.** This mode drops all historical receipts and bodies, but it cannot be used for validation. -For more details, see [sync modes](../fundamentals/03-sync-modes.md). +For more details, see [sync modes](../fundamentals/sync.md). #### Database size with different configurations