Reorg fundamentals

NethermindEth · Sep 2, 2023 · 95a3630 · 95a3630
1 parent 5562405
commit 95a3630
Show file tree

Hide file tree

Showing 19 changed files with 184 additions and 213 deletions.
diff --git a/docs/07-resources/faq.md b/docs/07-resources/faq.md
@@ -55,17 +55,17 @@ If you have any issues, please reach out to us on Discord: [https://discord.gg/X
 
 ## Can I disable logging to file?
 
-You can find more details on the logging config page [here](../fundamentals/06-logs/logging-configuration.md)
+You can find more details on the logging config page [here](../fundamentals/logs.md)
 
 ## Can I disable logging of JSON RPC calls?
 
-You can find more details on the logging config page [here](../fundamentals/06-logs/logging-configuration.md)
+You can find more details on the logging config page [here](../fundamentals/logs.md)
 
 ## How can I configure validator on AuRa / Clique?
 
 You can find more details on running validators in the docs ->
 for [AuRa](./guides-and-helpers/validator-setup/aura-validator.md) and
-for [Clique](../fundamentals/09-private-networks/how-to-setup-a-nethermind-only-clique-based-chain.md).
+for [Clique](../fundamentals/private-networks/clique.md).
 
 You can learn more about how to setup a aura validator [here](./guides-and-helpers/validator-setup/aura-validator.md)
 

diff --git a/docs/fundamentals/06-logs/_category_.json b/docs/fundamentals/06-logs/_category_.json
diff --git a/docs/fundamentals/06-logs/logging-configuration.md b/docs/fundamentals/06-logs/logging-configuration.md
diff --git a/docs/fundamentals/07-database.md b/docs/fundamentals/07-database.md
diff --git a/docs/fundamentals/08-how-to-reduce-database-size/_category_.json b/docs/fundamentals/08-how-to-reduce-database-size/_category_.json
diff --git a/docs/fundamentals/08-how-to-reduce-database-size/resync-database-from-scratch.md b/docs/fundamentals/08-how-to-reduce-database-size/resync-database-from-scratch.md
diff --git a/docs/fundamentals/09-private-networks/_category_.json b/docs/fundamentals/09-private-networks/_category_.json
diff --git a/docs/fundamentals/configuration.md b/docs/fundamentals/configuration.md
@@ -72,7 +72,7 @@ The command line options are case-sensitive and can be defined only once unless
   </p>
   </details>
 
-- **`-c, --configsDirectory <path>`**
+- **`-cd, --configsDirectory <path>`**
 
   An absolute or relative path to the configuration files directory. Defaults to `configs`.
 

diff --git a/.../08-how-to-reduce-database-size/README.md → docs/fundamentals/database.md b/.../08-how-to-reduce-database-size/README.md → docs/fundamentals/database.md
@@ -1,8 +1,35 @@
 ---
-description: Description of all possible options, their pros and cons
+title: Database
+sidebar_position: 5
 ---
 
-# How to reduce database size
+Nethermind uses the [RocksDB](https://rocksdb.org) database to store state. By default the database is stored in the
+same directory where the client binaries are. You can change it by providing a `--baseDbPath` config switch in the
+command line, e.g.`./Nethermind.Runner --config goerli --baseDbPath /home/username/nethermind_db`
+
+After Nethermind is started, you will see multiple directories appearing in the _baseDbPath_ directory.
+
+![Example of the DB directory on a freshly deployed Ubuntu VM with Nethermind.](</img/image(61).png>)
+
+| DB Directory      | Content                                                                                                                                  |
+|-------------------|------------------------------------------------------------------------------------------------------------------------------------------|
+| blockInfos        | information about blocks at each level of the block tree (canonical chain and branches)                                                  |
+| blocks            | block bodies (block transactions and uncle data)                                                                                         |
+| bloom             | bloom indexes for fast log searches                                                                                                      |
+| canonicalHashTrie | LES protocol related data                                                                                                                |
+| code              | contract bytecodes                                                                                                                       |
+| discoveryNodes    | peers discovered via discovery protocol - used for quick peering after restarts (you can copy this DB between nodes to speed up peering) |
+| headers           | block headers only                                                                                                                       |
+| pendingTx         | (this DB is wiped out on each restart) 2nd level cache of pending transactions / mempool (1st level is in memory)                        |
+| peers             | stores additional sync peers information (like peer reputation) - you can copy this DB between nodes to speed up peering on fresh sync   |
+| receipts          | transaction receipts data                                                                                                                |
+| state             | blockchain state including accounts and contract storage (Patricia trie nodes)                                                           |
+
+You can use `rsync` between your nodes to clone the database (One of our users copied entire 4.5TB archive state this
+way while the node was running and only stopped the node for the very last stage of `rsync` ). You can also simply copy
+the database between Unix and Windows systems (and most likely macOS).
+
+## How to reduce database size
 
 The Nethermind database can experience substantial growth over time, starting from an initial size of approximately 650
 GB. As a result, many node-setups are configured to run on 1TB disks. However, even with the application of settings
@@ -11,7 +38,7 @@ designed to slow the growth rate, these disks may eventually run out of free spa
 Current options to reduce db size are:
 
 1. [Re-sync database from scratch](resync-database-from-scratch.md)
-2. [Full pruning](full-pruning.md)
+2. [Full pruning](../pruning.md)
 
 The table below presents a short comparison of these methods including possible fine-tuning of each method. Data was
 fetched from a node running on a machine with the below specifications:\

diff --git a/docs/fundamentals/06-logs/logs.md → docs/fundamentals/logs.md b/docs/fundamentals/06-logs/logs.md → docs/fundamentals/logs.md
@@ -1,11 +1,65 @@
 ---
-description: A quick start example for Görli Testnet Nethermind synchronization
+title: Logs
+sidebar_position: 4
 ---
 
-# Explaining Nethermind logs
+## Log config file location
+
+Logging in Nethermind is done via NLog library that can be configured by editing the NLog.config file.
+
+| Environment Type                                                                  | NLog.config location                                                   |
+|-----------------------------------------------------------------------------------|------------------------------------------------------------------------|
+| built from src - Debug mode                                                       | src\Nethermind\Nethermind.Runner\bin\Debug\netcoreapp3.1\NLog.config   |
+| built from src - Release mode                                                     | src\Nethermind\Nethermind.Runner\bin\Release\netcoreapp3.1\NLog.config |
+| PPA                                                                               | /usr/share/nethermind/NLog.config                                      |
+| Docker                                                                            | /nethermind/NLog.config                                                |
+| from [downloads page](https://downloads.nethermind.io)                            | top level directory after unzipping the package                        |
+| from [GitHub releases page](https://github.com/NethermindEth/nethermind/releases) | top level directory after unzipping the package                        |
+| dAppNode                                                                          | ? \[to be documented]                                                  |
+
+## Log config file syntax
+
+Detailed NLog configuration options can be found
+here: [https://nlog-project.org/config/](https://nlog-project.org/config/)
+
+## Config or CLI log rules
+
+Simple logging rules can be added through configuration file or command line argument.
+
+For example this would add `Trace` level logs to any logger under `Synchronization` module and `Debug` level logs
+for `BlockTree` from `Blockchain` module:\
+`--Init.LogRules Synchronization.*:Trace;Blockchain.BlockTree:Debug`
+
+## Global logging override
+
+Additionally there are global logging override that you can use temporarily:
+
+| Command line override                            | Log level |
+|--------------------------------------------------|-----------|
+| ./Nethermind.Runner --config mainnet --log TRACE | TRACE     |
+| ./Nethermind.Runner --config mainnet --log DEBUG | DEBUG     |
+| ./Nethermind.Runner --config mainnet --log INFO  | INFO      |
+| ./Nethermind.Runner --config mainnet --log WARN  | WARN      |
+| ./Nethermind.Runner --config mainnet --log ERROR | ERROR     |
+
+## JSON RPC logging level
+
+This can be done by including these lines in the logging configuration file:
+
+```
+<logger name="JsonRpc.*" minlevel="Error" writeTo="file-async"/>
+<logger name="JsonRpc.*" minlevel="Error" writeTo="auto-colored-console-async" final="true"/>
+<logger name="JsonRpc.*" final="true"/>
+```
+
+## Enterprise Logging
+
+See how to configure Seq [here](https://docs.nethermind.io/nethermind/enterprise/seq)
+
+## Explaining Nethermind logs
 
 You can check the supported operating systems, architectures and hardware requirements
-here: [system-requirements.md](../../get-started/system-requirements.md)
+here: [system-requirements.md](../get-started/system-requirements.md)
 
 After the node starts, you will see some initial info about the node and then the sync will start. Görli fast sync uses
 a `fast blocks` sync mode initially. The `fast blocks` sync picks some known `pivot block` from the past and
@@ -85,4 +139,4 @@ Also, every now and then, a peer report will appear like below:
     * Receipts Transfer
     * Node Data Transfer
     * Snap Ranges Transfer
-5. Fifth bracket is for Client Info like Client Name, Client Version, Operating System and Language Version.
+5. Fifth bracket is for Client Info like Client Name, Client Version, Operating System and Language Version.
diff --git a/docs/fundamentals/private-networks/_category_.json b/docs/fundamentals/private-networks/_category_.json
@@ -0,0 +1,9 @@
+{
+  "position": 6,
+  "label": "Private networks",
+  "collapsible": true,
+  "link": {
+    "type": "doc",
+    "id": "overview"
+  }
+}