diff --git a/config/profile.go b/config/profile.go index 7c7e3007835..8d3e306aa44 100644 --- a/config/profile.go +++ b/config/profile.go @@ -135,7 +135,11 @@ You should use this datastore if: * You want to minimize memory usage. * You are ok with the default speed of data import, or prefer to use --nocopy. -This profile may only be applied when first initializing the node. +See configuration documentation at: +https://github.com/ipfs/kubo/blob/master/docs/datastores.md#flatfs + +NOTE: This profile may only be applied when first initializing node at IPFS_PATH + via 'ipfs init --profile flatfs' `, InitOnly: true, @@ -153,11 +157,16 @@ You should use this datastore if: - You need a datastore that is focused on performance. - You need reliability by default, but may choose to disable WAL for maximum performance when reliability is not critical. -- This datastore is good for multi-terrabyte data sets. -- May benefit from tuning depeending on read/write patterns and throughput. -- Performance is helped significnatly by running on a system with plenty of memory. +- This datastore is good for multi-terabyte data sets. +- May benefit from tuning depending on read/write patterns and throughput. +- Performance is helped significantly by running on a system with plenty of memory. + +See configuration documentation at: +https://github.com/ipfs/kubo/blob/master/docs/datastores.md#pebbleds -This profile may only be applied when first initializing the node.`, +NOTE: This profile may only be applied when first initializing node at IPFS_PATH + via 'ipfs init --profile pebbleds' +`, InitOnly: true, Transform: func(c *Config) error { @@ -181,7 +190,12 @@ Other caveats: * Good for medium-size datastores, but may run into performance issues if your dataset is bigger than a terabyte. -This profile may only be applied when first initializing the node.`, +See configuration documentation at: +https://github.com/ipfs/kubo/blob/master/docs/datastores.md#badgerds + +NOTE: This profile may only be applied when first initializing node at IPFS_PATH + via 'ipfs init --profile badgerds' +`, InitOnly: true, Transform: func(c *Config) error { diff --git a/docs/changelogs/v0.31.md b/docs/changelogs/v0.31.md index af59ed765ac..5cdf4d6d201 100644 --- a/docs/changelogs/v0.31.md +++ b/docs/changelogs/v0.31.md @@ -6,7 +6,7 @@ - [Overview](#overview) - [๐Ÿ”ฆ Highlights](#-highlights) - - [Include pebble as built-in plugin](#pebble-plugin) + - [Experimental Pebble Datastore](#experimental-pebble-datastore) - [๐Ÿ“ Changelog](#-changelog) - [๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆ Contributors](#-contributors) @@ -14,13 +14,14 @@ ### ๐Ÿ”ฆ Highlights -#### Pebble Plugin +#### Experimental Pebble Datastore -Pebble provides a high-performance alternative to leveldb as the datastore, and will serve as a replacement for badger1. +[Pebble](https://github.com/ipfs/kubo/blob/master/docs/config.md#pebbleds-profile) provides a high-performance alternative to leveldb as the datastore, and provides a modern replacement for [legacy badgerv1](https://github.com/ipfs/kubo/blob/master/docs/config.md#badgerds-profile). -There are a number of tuning parameters available for tuning pebble's performance to your specific needs. Default values are used for any parameters that are not configured or are set to their zero-value. +A fresh Kubo node can be initialized with [`pebbleds` profile](https://github.com/ipfs/kubo/blob/master/docs/config.md#pebbleds-profile) via `ipfs init --profile pebbleds`. -For a description of the available tuning parameters, see: https://github.com/ipfs/kubo/blob/master/docs/datastores.md#pebbleds +There are a number of parameters available for tuning pebble's performance to your specific needs. Default values are used for any parameters that are not configured or are set to their zero-value. +For a description of the available tuning parameters, see [kubo/docs/datastores.md#pebbleds](https://github.com/ipfs/kubo/blob/master/docs/datastores.md#pebbleds). ### ๐Ÿ“ Changelog diff --git a/docs/config.md b/docs/config.md index 3c11d586410..b4473815436 100644 --- a/docs/config.md +++ b/docs/config.md @@ -523,8 +523,8 @@ Spec defines the structure of the ipfs datastore. It is a composable structure, where each datastore is represented by a json object. Datastores can wrap other datastores to provide extra functionality (eg metrics, logging, or caching). -For more information on possible values for this configuration option, see -[docs/datastores.md](datastores.md) +> [!NOTE] +> For more information on possible values for this configuration option, see [`kubo/docs/datastores.md`](datastores.md) Default: ``` @@ -2397,9 +2397,9 @@ Inverse profile of the test profile. ### `flatfs` profile -Configures the node to use the flatfs datastore. Flatfs is the default datastore. +Configures the node to use the flatfs datastore. +Flatfs is the default, most battle-tested and reliable datastore. -This is the most battle-tested and reliable datastore. You should use this datastore if: - You need a very simple and very reliable datastore, and you trust your @@ -2410,7 +2410,11 @@ You should use this datastore if: - You want to minimize memory usage. - You are ok with the default speed of data import, or prefer to use `--nocopy`. -This profile may only be applied when first initializing the node. +> [!WARNING] +> This profile may only be applied when first initializing the node via `ipfs init --profile flatfs` + +> [!NOTE] +> See caveats and configuration options at [`datastores.md#flatfs`](datastores.md#flatfs) ### `pebbleds` profile @@ -2425,7 +2429,11 @@ You should use this datastore if: - May benefit from tuning depeending on read/write patterns and throughput. - Performance is helped significnatly by running on a system with plenty of memory. -This profile may only be applied when first initializing the node. +> [!WARNING] +> This profile may only be applied when first initializing the node via `ipfs init --profile pebbleds` + +> [!NOTE] +> See other caveats and configuration options at [`datastores.md#pebbleds`](datastores.md#pebbleds) ### `badgerds` profile @@ -2446,7 +2454,11 @@ Also, be aware that: - Good for medium-size datastores, but may run into performance issues if your dataset is bigger than a terabyte. - The current implementation is based on old badger 1.x which is no longer supported by the upstream team. -This profile may only be applied when first initializing the node. +> [!WARNING] +> This profile may only be applied when first initializing the node via `ipfs init --profile badgerds` + +> [!NOTE] +> See other caveats and configuration options at [`datastores.md#pebbleds`](datastores.md#pebbleds) ### `lowpower` profile diff --git a/docs/datastores.md b/docs/datastores.md index 63eb797f1f2..f2051b601d4 100644 --- a/docs/datastores.md +++ b/docs/datastores.md @@ -3,6 +3,13 @@ This document describes the different possible values for the `Datastore.Spec` field in the ipfs configuration file. +- [flatfs](#flatfs) +- [levelds](#levelds) +- [pebbleds](#pebbleds) +- [badgerds](#badgerds) +- [mount](#mount) +- [measure](#measure) + ## flatfs Stores each key value pair as a file on the filesystem. @@ -39,7 +46,15 @@ Uses a leveldb database to store key value pairs. Uses [pebble](https://github.com/cockroachdb/pebble) as a key value store. -#### The following options are availble for tuning pebble. If they are not configured (or assigned their zero-valued), then default values are used. Start using pebble with only default values and configure tuning items are needed for your needs. For a more complete description of these values, see: https://pkg.go.dev/github.com/cockroachdb/pebble@v1.1.2#Options +```json +{ + "type": "pebbleds", + "path": "", +} +``` + +The following options are availble for tuning pebble. +If they are not configured (or assigned their zero-valued), then default values are used. * `bytesPerSync`: int, Sync sstables periodically in order to smooth out writes to disk. (default: 512KB) * `bisableWAL`: true|false, Disable the write-ahead log (WAL) at expense of prohibiting crash recovery. (default: false) @@ -53,13 +68,8 @@ Uses [pebble](https://github.com/cockroachdb/pebble) as a key value store. * `walBytesPerSync`: int: Sets the number of bytes to write to a WAL before calling Sync on it in the background. (default: 0, no background syncing) * `walMinSyncSeconds`: int: Sets the minimum duration between syncs of the WAL. (default: 0) - -```json -{ - "type": "pebbleds", - "path": "", -} -``` +> [!TIP] +> Start using pebble with only default values and configure tuning items are needed for your needs. For a more complete description of these values, see: `https://pkg.go.dev/github.com/cockroachdb/pebble@vA.B.C#Options` (where `A.B.C` is pebble version from Kubo's `go.mod`). ## badgerds