From b26e1dfd4d41005af260e0583c26b1d7e6f32942 Mon Sep 17 00:00:00 2001 From: hlcianfagna <110453267+hlcianfagna@users.noreply.github.com> Date: Mon, 26 Feb 2024 17:55:32 +0100 Subject: [PATCH 1/2] Administration/Clustering: Add section about "scaling clusters up and down" --- docs/admin/clustering/index.rst | 1 + docs/admin/clustering/scale-up-down.md | 40 ++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) create mode 100644 docs/admin/clustering/scale-up-down.md diff --git a/docs/admin/clustering/index.rst b/docs/admin/clustering/index.rst index f5278209..2f5b4883 100644 --- a/docs/admin/clustering/index.rst +++ b/docs/admin/clustering/index.rst @@ -14,5 +14,6 @@ adding more nodes. This section of the documentation covers that topic. multi-node-setup multi-zone-setup + scale-up-down kubernetes logical-replication-setup diff --git a/docs/admin/clustering/scale-up-down.md b/docs/admin/clustering/scale-up-down.md new file mode 100644 index 00000000..97bb2684 --- /dev/null +++ b/docs/admin/clustering/scale-up-down.md @@ -0,0 +1,40 @@ +(scaling-clusters)= + +# Scaling Clusters Up and Down + +The article about [scaling CrateDB clusters up and down to cope with peaks in +demand] shares knowledge about the [shard allocation filtering] feature of +CrateDB. + +Along the lines, it demonstrates how this functionality is applied in a real- +world data management scenario, which is about tuning your database cluster to +cope with high-demand situations. + +Prepare adding extra nodes to the database cluster. +```sql +/* Apply routing setting to all existing partitions and new partitions. */ +ALTER TABLE test SET ("routing.allocation.exclude.storage" = 'temporarynodes'); + +/* Configure the setting to be excluded / not applied to _new_ partitions. */ +ALTER TABLE ONLY test RESET ("routing.allocation.exclude.storage"); +``` + +Before the high-demand event, properly configure table routing accordingly. +```sql +ALTER TABLE ONLY test SET ("routing.allocation.total_shards_per_node" = 2); +``` + +To decommission extra database nodes, we need to move the data collected during +the days of the event. +```sql +-- Move the collected data off the extra nodes. +ALTER TABLE test SET ("routing.allocation.exclude.storage" = 'temporarynodes'); +ALTER TABLE test RESET ("routing.allocation.total_shards_per_node"); + +-- Invoke the decommissioning. +ALTER CLUSTER DECOMMISSION 'nodename'; +``` + + +[scaling CrateDB clusters up and down to cope with peaks in demand]: https://community.cratedb.com/t/scaling-cratedb-clusters-up-and-down-to-cope-with-peaks-in-demand/1314 +[shard allocation filtering]: inv:crate-reference#ddl_shard_allocation From 8198f08ccc82ba12c94833235f5c9695513dee09 Mon Sep 17 00:00:00 2001 From: hlcianfagna <110453267+hlcianfagna@users.noreply.github.com> Date: Tue, 27 Feb 2024 00:53:08 +0100 Subject: [PATCH 2/2] Administration/Clustering: Add section about "expanding clusters" ... linking to community forum article "How to add new nodes to an existing cluster". -- https://community.cratedb.com/t/how-to-add-new-nodes-to-an-existing-cluster/1546 --- docs/admin/clustering/scale-up-down.md | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/docs/admin/clustering/scale-up-down.md b/docs/admin/clustering/scale-up-down.md index 97bb2684..9654441c 100644 --- a/docs/admin/clustering/scale-up-down.md +++ b/docs/admin/clustering/scale-up-down.md @@ -1,7 +1,28 @@ (scaling-clusters)= - # Scaling Clusters Up and Down +A significant feature in CrateDB is that it can scale horizontally, which means +that instead of adding more RAM, CPU, and disk resources to existing nodes, you +will add more individual nodes to your CrateDB cluster. + +By running your database cluster on multiple nodes, you will gain two benefits. + +- Store data volumes larger than being able to be handled on a single machine. +- Add resiliency to your distributed database cluster, by increasing the number + of replica nodes. + + +(scaling-expand)= +## Expand Cluster + +The article about [how to add new nodes to an existing cluster] walks you +through the process of scaling up your database cluster, and educates you +about the corresponding details to consider. + + +(scaling-ondemand)= +## On-Demand Scaling + The article about [scaling CrateDB clusters up and down to cope with peaks in demand] shares knowledge about the [shard allocation filtering] feature of CrateDB. @@ -36,5 +57,6 @@ ALTER CLUSTER DECOMMISSION 'nodename'; ``` +[how to add new nodes to an existing cluster]: https://community.cratedb.com/t/how-to-add-new-nodes-to-an-existing-cluster/1546 [scaling CrateDB clusters up and down to cope with peaks in demand]: https://community.cratedb.com/t/scaling-cratedb-clusters-up-and-down-to-cope-with-peaks-in-demand/1314 [shard allocation filtering]: inv:crate-reference#ddl_shard_allocation