docs: pebbleds profile

ipfs · Oct 2, 2024 · a868e08 · a868e08
1 parent 6a5dae8
commit a868e08
Show file tree

Hide file tree

Showing 4 changed files with 63 additions and 26 deletions.
diff --git a/config/profile.go b/config/profile.go
@@ -135,7 +135,11 @@ You should use this datastore if:
 * You want to minimize memory usage.
 * You are ok with the default speed of data import, or prefer to use --nocopy.
 
-This profile may only be applied when first initializing the node.
+See configuration documentation at:
+https://github.com/ipfs/kubo/blob/master/docs/datastores.md#flatfs
+
+NOTE: This profile may only be applied when first initializing node at IPFS_PATH
+      via 'ipfs init --profile flatfs'
 `,
 
 		InitOnly: true,
@@ -153,11 +157,16 @@ You should use this datastore if:
 
 - You need a datastore that is focused on performance.
 - You need reliability by default, but may choose to disable WAL for maximum performance when reliability is not critical.
-- This datastore is good for multi-terrabyte data sets.
-- May benefit from tuning depeending on read/write patterns and throughput.
-- Performance is helped significnatly by running on a system with plenty of memory.
+- This datastore is good for multi-terabyte data sets.
+- May benefit from tuning depending on read/write patterns and throughput.
+- Performance is helped significantly by running on a system with plenty of memory.
+
+See configuration documentation at:
+https://github.com/ipfs/kubo/blob/master/docs/datastores.md#pebbleds
 
-This profile may only be applied when first initializing the node.`,
+NOTE: This profile may only be applied when first initializing node at IPFS_PATH
+      via 'ipfs init --profile pebbleds'
+`,
 
 		InitOnly: true,
 		Transform: func(c *Config) error {
@@ -181,7 +190,12 @@ Other caveats:
 * Good for medium-size datastores, but may run into performance issues
   if your dataset is bigger than a terabyte.
 
-This profile may only be applied when first initializing the node.`,
+See configuration documentation at:
+https://github.com/ipfs/kubo/blob/master/docs/datastores.md#badgerds
+
+NOTE: This profile may only be applied when first initializing node at IPFS_PATH
+      via 'ipfs init --profile badgerds'
+`,
 
 		InitOnly: true,
 		Transform: func(c *Config) error {

diff --git a/docs/changelogs/v0.31.md b/docs/changelogs/v0.31.md
@@ -6,21 +6,22 @@
 
 - [Overview](#overview)
 - [🔦 Highlights](#-highlights)
-  - [Include pebble as built-in plugin](#pebble-plugin)
+  - [Experimental Pebble Datastore](#experimental-pebble-datastore)
 - [📝 Changelog](#-changelog)
 - [👨‍👩‍👧‍👦 Contributors](#-contributors)
 
 ### Overview
 
 ### 🔦 Highlights
 
-#### Pebble Plugin
+#### Experimental Pebble Datastore
 
-Pebble provides a high-performance alternative to leveldb as the datastore, and will serve as a replacement for badger1.
+[Pebble](https://github.com/ipfs/kubo/blob/master/docs/config.md#pebbleds-profile) provides a high-performance alternative to leveldb as the datastore, and provides a modern replacement for [legacy badgerv1](https://github.com/ipfs/kubo/blob/master/docs/config.md#badgerds-profile).
 
-There are a number of tuning parameters available for tuning pebble's performance to your specific needs. Default values are used for any parameters that are not configured or are set to their zero-value.
+A fresh Kubo node can be initialized with [`pebbleds` profile](https://github.com/ipfs/kubo/blob/master/docs/config.md#pebbleds-profile) via `ipfs init --profile pebbleds`.
 
-For a description of the available tuning parameters, see: https://github.com/ipfs/kubo/blob/master/docs/datastores.md#pebbleds
+There are a number of parameters available for tuning pebble's performance to your specific needs. Default values are used for any parameters that are not configured or are set to their zero-value.
+For a description of the available tuning parameters, see [kubo/docs/datastores.md#pebbleds](https://github.com/ipfs/kubo/blob/master/docs/datastores.md#pebbleds).
 
 ### 📝 Changelog
 

diff --git a/docs/config.md b/docs/config.md
@@ -523,8 +523,8 @@ Spec defines the structure of the ipfs datastore. It is a composable structure,
 where each datastore is represented by a json object. Datastores can wrap other
 datastores to provide extra functionality (eg metrics, logging, or caching).
 
-For more information on possible values for this configuration option, see
-[docs/datastores.md](datastores.md)
+> [!NOTE]
+> For more information on possible values for this configuration option, see [`kubo/docs/datastores.md`](datastores.md)
 
 Default:
 ```
@@ -2397,9 +2397,9 @@ Inverse profile of the test profile.
 
 ### `flatfs` profile
 
-Configures the node to use the flatfs datastore. Flatfs is the default datastore.
+Configures the node to use the flatfs datastore.
+Flatfs is the default, most battle-tested and reliable datastore.
 
-This is the most battle-tested and reliable datastore.
 You should use this datastore if:
 
 - You need a very simple and very reliable datastore, and you trust your
@@ -2410,7 +2410,11 @@ You should use this datastore if:
 - You want to minimize memory usage.
 - You are ok with the default speed of data import, or prefer to use `--nocopy`.
 
-This profile may only be applied when first initializing the node.
+> [!WARNING]
+> This profile may only be applied when first initializing the node via `ipfs init --profile flatfs`
+
+> [!NOTE]
+> See caveats and configuration options at [`datastores.md#flatfs`](datastores.md#flatfs)
 
 ### `pebbleds` profile
 
@@ -2425,7 +2429,11 @@ You should use this datastore if:
 - May benefit from tuning depeending on read/write patterns and throughput.
 - Performance is helped significnatly by running on a system with plenty of memory.
 
-This profile may only be applied when first initializing the node.
+> [!WARNING]
+> This profile may only be applied when first initializing the node via `ipfs init --profile pebbleds`
+
+> [!NOTE]
+> See other caveats and configuration options at [`datastores.md#pebbleds`](datastores.md#pebbleds)
 
 ### `badgerds` profile
 
@@ -2446,7 +2454,11 @@ Also, be aware that:
 - Good for medium-size datastores, but may run into performance issues if your dataset is bigger than a terabyte.
 - The current implementation is based on old badger 1.x which is no longer supported by the upstream team.
 
-This profile may only be applied when first initializing the node.
+> [!WARNING]
+> This profile may only be applied when first initializing the node via `ipfs init --profile badgerds`
+
+> [!NOTE]
+> See other caveats and configuration options at [`datastores.md#pebbleds`](datastores.md#pebbleds)
 
 ### `lowpower` profile
 

diff --git a/docs/datastores.md b/docs/datastores.md
@@ -3,6 +3,13 @@
 This document describes the different possible values for the `Datastore.Spec`
 field in the ipfs configuration file.
 
+- [flatfs](#flatfs)
+- [levelds](#levelds)
+- [pebbleds](#pebbleds)
+- [badgerds](#badgerds)
+- [mount](#mount)
+- [measure](#measure)
+
 ## flatfs
 
 Stores each key value pair as a file on the filesystem.
@@ -39,7 +46,15 @@ Uses a leveldb database to store key value pairs.
 
 Uses [pebble](https://github.com/cockroachdb/pebble) as a key value store.
 
-#### The following options are availble for tuning pebble. If they are not configured (or assigned their zero-valued), then default values are used. Start using pebble with only default values and configure tuning items are needed for your needs. For a more complete description of these values, see: https://pkg.go.dev/github.com/cockroachdb/[email protected]#Options
+```json
+{
+	"type": "pebbleds",
+	"path": "<location of pebble inside repo>",
+}
+```
+
+The following options are availble for tuning pebble.
+If they are not configured (or assigned their zero-valued), then default values are used.
 
 * `bytesPerSync`: int, Sync sstables periodically in order to smooth out writes to disk. (default: 512KB)
 * `bisableWAL`: true|false, Disable the write-ahead log (WAL) at expense of prohibiting crash recovery. (default: false)
@@ -53,13 +68,8 @@ Uses [pebble](https://github.com/cockroachdb/pebble) as a key value store.
 * `walBytesPerSync`: int: Sets the number of bytes to write to a WAL before calling Sync on it in the background. (default: 0, no background syncing)
 * `walMinSyncSeconds`: int: Sets the minimum duration between syncs of the WAL. (default: 0)
 
-
-```json
-{
-	"type": "pebbleds",
-	"path": "<location of pebble inside repo>",
-}
-```
+> [!TIP]
+> Start using pebble with only default values and configure tuning items are needed for your needs. For a more complete description of these values, see: `https://pkg.go.dev/github.com/cockroachdb/[email protected]#Options` (where `A.B.C` is pebble version from Kubo's `go.mod`).
 
 ## badgerds