-
Notifications
You must be signed in to change notification settings - Fork 70
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: yandongxiao <[email protected]>
- Loading branch information
1 parent
06af429
commit 39d3274
Showing
13 changed files
with
387 additions
and
38 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -34,3 +34,4 @@ gen-crd-api-reference-docs-master/ | |
| *.tgz | ||
|
|
||
| temp/ | ||
| cmd/cmd | ||
269 changes: 269 additions & 0 deletions
269
doc/setup-starrocks-when-readOnlyRootFilesystem-is-true.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,269 @@ | ||
| # Background | ||
|
|
||
| StarRocks has three components: Frontend (FE), Backend (BE), and Compute Node(CN). When the `readOnlyRootFilesystem` is | ||
| set to `true`, the components of StarRocks cannot start normally. This is because the components of StarRocks write data | ||
| to the disk, and the `readOnlyRootFilesystem` setting prevents the components from writing data to the disk. | ||
|
|
||
| For the FE component, FE writes data to the following directories: | ||
|
|
||
| ```bash | ||
| # in fe directory | ||
| drwxr-xr-x 2 root root 4.0K Nov 19 11:27 plugins | ||
| drwxr-xr-x 4 root root 4.0K Nov 19 11:27 temp_dir | ||
|
|
||
| # in fe/bin directory | ||
| -rw-r--r-- 1 root root 2 Nov 19 11:27 fe.pid | ||
|
|
||
| # in fe/conf directory | ||
| lrwxrwxrwx 1 root root 30 Nov 19 11:27 fe.conf -> /etc/starrocks/fe/conf/fe.conf | ||
| ``` | ||
|
|
||
| For the BE component, BE writes data to the following directories: | ||
|
|
||
| ```bash | ||
| # in be directory | ||
| drwxr-xr-x 2 root root 4.0K Nov 19 11:27 spill | ||
|
|
||
| # in be/conf directory | ||
| lrwxrwxrwx 1 root root 30 Nov 19 11:27 be.conf -> /etc/starrocks/be/conf/be.conf | ||
|
|
||
| # in be/bin directory | ||
| -rw-r----- 1 root root 3 Nov 19 11:27 be.pid | ||
|
|
||
| # in be/lib directory | ||
| drwxr-xr-x 2 root root 4.0K Nov 19 11:27 jdbc_drivers | ||
| drwxr-xr-x 2 root root 4.0K Nov 19 11:27 small_file | ||
| drwxr-xr-x 130 root root 4.0K Nov 19 11:27 udf | ||
| drwxr-xr-x 2 root root 4.0K Nov 19 11:27 udf-runtime | ||
| ``` | ||
|
|
||
| This document describes how to set up StarRocks when the `readOnlyRootFilesystem` field is set to `true`. | ||
|
|
||
| # How | ||
|
|
||
| We create and mount a volume, and in the entrypoint script, we will copy everything from the original directory to the | ||
| mounted volume. This way, the components of StarRocks can write data to the mounted volume. | ||
|
|
||
| > Note: you should use the operator version `v1.9.9` or later. | ||
| # Steps | ||
|
|
||
| There are two ways to deploy StarRocks cluster: | ||
|
|
||
| 1. Deploy StarRocks cluster with `StarRocksCluster` CR yaml. | ||
| 2. Deploy StarRocks cluster with Helm chart. | ||
|
|
||
| Therefore, there are two ways to set up StarRocks when the `readOnlyRootFilesystem` field is set to `true`. | ||
|
|
||
| ## By using StarRocksCluster CR yaml | ||
|
|
||
| ```yaml | ||
| apiVersion: starrocks.com/v1 | ||
| kind: StarRocksCluster | ||
| metadata: | ||
| name: kube-starrocks | ||
| namespace: starrocks | ||
| spec: | ||
| starRocksFeSpec: | ||
| command: [ "bash", "-c" ] | ||
| args: | ||
| - cp -r /opt/starrocks/* /opt/starrocks-artifacts && export STARROCKS_ROOT=/opt/starrocks-artifacts && exec /opt/starrocks-artifacts/fe_entrypoint.sh $FE_SERVICE_NAME | ||
| readOnlyRootFilesystem: true | ||
| configMapInfo: | ||
| configMapName: kube-starrocks-fe-cm | ||
| resolveKey: fe.conf | ||
| image: starrocks/fe-ubuntu:3.2.2 | ||
| imagePullPolicy: IfNotPresent | ||
| replicas: 1 | ||
| requests: | ||
| cpu: 1m | ||
| memory: 22Mi | ||
| storageVolumes: | ||
| - mountPath: /opt/starrocks-artifacts | ||
| name: fe-artifacts | ||
| storageClassName: emptyDir | ||
| storageSize: 20Gi | ||
| - mountPath: /opt/starrocks-meta | ||
| name: fe-meta | ||
| storageSize: 10Gi | ||
| - mountPath: /opt/starrocks-log | ||
| name: fe-log | ||
| storageSize: 10Gi | ||
| starRocksBeSpec: | ||
| command: [ "bash", "-c" ] | ||
| args: | ||
| - cp -r /opt/starrocks/* /opt/starrocks-artifacts && export STARROCKS_ROOT=/opt/starrocks-artifacts && exec /opt/starrocks-artifacts/be_entrypoint.sh $FE_SERVICE_NAME | ||
| readOnlyRootFilesystem: true | ||
| configMapInfo: | ||
| configMapName: kube-starrocks-be-cm | ||
| resolveKey: be.conf | ||
| image: starrocks/be-ubuntu:3.2.2 | ||
| imagePullPolicy: IfNotPresent | ||
| replicas: 1 | ||
| requests: | ||
| cpu: 1m | ||
| memory: 10Mi | ||
| storageVolumes: | ||
| - mountPath: /opt/starrocks-artifacts | ||
| name: be-artifacts | ||
| storageClassName: emptyDir | ||
| storageSize: 20Gi | ||
| - mountPath: /opt/starrocks-storage | ||
| name: be-storage # the name must be this | ||
| storageSize: 10Gi | ||
| - mountPath: /opt/starrocks-log | ||
| name: be-log # the name must be this | ||
| storageSize: 10Gi | ||
|
|
||
| --- | ||
|
|
||
| apiVersion: v1 | ||
| data: | ||
| fe.conf: | | ||
| LOG_DIR = ${STARROCKS_HOME}/log | ||
| DATE = "$(date +%Y%m%d-%H%M%S)" | ||
| JAVA_OPTS="-Dlog4j2.formatMsgNoLookups=true -Xmx8192m -XX:+UseG1GC -Xlog:gc*:${LOG_DIR}/fe.gc.log.$DATE:time" | ||
| http_port = 8030 | ||
| rpc_port = 9020 | ||
| query_port = 9030 | ||
| edit_log_port = 9010 | ||
| mysql_service_nio_enabled = true | ||
| sys_log_level = INFO | ||
| # config for meta and log | ||
| meta_dir = /opt/starrocks-meta | ||
| dump_log_dir = /opt/starrocks-log | ||
| sys_log_dir = /opt/starrocks-log | ||
| audit_log_dir = /opt/starrocks-log | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: kube-starrocks-fe-cm | ||
| namespace: starrocks | ||
|
|
||
| --- | ||
|
|
||
| apiVersion: v1 | ||
| data: | ||
| be.conf: | | ||
| be_port = 9060 | ||
| webserver_port = 8040 | ||
| heartbeat_service_port = 9050 | ||
| brpc_port = 8060 | ||
| sys_log_level = INFO | ||
| default_rowset_type = beta | ||
| # config for storage and log | ||
| storage_root_path = /opt/starrocks-storage | ||
| sys_log_dir = /opt/starrocks-log | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: kube-starrocks-be-cm | ||
| namespace: starrocks | ||
| ``` | ||
| ## By using Helm Chart | ||
| If you are using the `kube-starrocks` Helm chart, add the following snippets to `values.yaml`. | ||
|
|
||
| > Note: you should use the chart version `v1.9.9` or later. | ||
|
|
||
| ```yaml | ||
| operator: | ||
| starrocksOperator: | ||
| image: | ||
| repository: operator | ||
| tag: v1.9.9-rc1 | ||
| imagePullPolicy: IfNotPresent | ||
| resources: | ||
| requests: | ||
| cpu: 1m | ||
| memory: 20Mi | ||
| starrocks: | ||
| initPassword: | ||
| enabled: false | ||
| password: "123456" | ||
| starrocksBeSpec: | ||
| readOnlyRootFilesystem: true | ||
| entrypoint: | ||
| script: | | ||
| #! /bin/bash | ||
| echo "do something before start BE" | ||
| cp -r /opt/starrocks/* /opt/starrocks-artifacts | ||
| export STARROCKS_ROOT=/opt/starrocks-artifacts | ||
| exec /opt/starrocks-artifacts/be_entrypoint.sh $FE_SERVICE_NAME | ||
| image: | ||
| repository: starrocks/be-ubuntu | ||
| tag: 3.2.2 | ||
| replicas: 1 | ||
| resources: | ||
| limits: | ||
| cpu: 8 | ||
| memory: 8Gi | ||
| requests: | ||
| cpu: 1m | ||
| memory: 10Mi | ||
| storageSpec: | ||
| name: be | ||
| storageSize: 10Gi | ||
| storageMountPath: /opt/starrocks-storage | ||
| logStorageSize: 10Gi | ||
| logMountPath: /opt/starrocks-log | ||
| spillStorageSize: 10Gi | ||
| spillMountPath: /opt/starrocks-spill | ||
| emptyDirs: | ||
| - name: be-artifacts | ||
| mountPath: /opt/starrocks-artifacts | ||
| config: | | ||
| be_port = 9060 | ||
| webserver_port = 8040 | ||
| heartbeat_service_port = 9050 | ||
| brpc_port = 8060 | ||
| sys_log_level = INFO | ||
| default_rowset_type = beta | ||
| # config for storage and log | ||
| storage_root_path = /opt/starrocks-storage | ||
| sys_log_dir = /opt/starrocks-log | ||
| spill_local_storage_dir = /opt/starrocks-spill | ||
| starrocksFESpec: | ||
| readOnlyRootFilesystem: true | ||
| entrypoint: | ||
| script: | | ||
| #! /bin/bash | ||
| cp -r /opt/starrocks/* /opt/starrocks-artifacts | ||
| export STARROCKS_ROOT=/opt/starrocks-artifacts | ||
| exec /opt/starrocks/fe_entrypoint.sh $FE_SERVICE_NAME | ||
| image: | ||
| repository: starrocks/fe-ubuntu | ||
| tag: 3.2.2 | ||
| resources: | ||
| limits: | ||
| cpu: 2 | ||
| memory: 4Gi | ||
| requests: | ||
| cpu: 1m | ||
| memory: 20Mi | ||
| storageSpec: | ||
| name: fe | ||
| storageSize: 10Gi | ||
| storageMountPath: /opt/starrocks-meta | ||
| logStorageSize: 10Gi | ||
| logMountPath: /opt/starrocks-log | ||
| emptyDirs: | ||
| - name: fe-artifacts | ||
| mountPath: /opt/starrocks-artifacts | ||
| config: | | ||
| LOG_DIR = ${STARROCKS_HOME}/log | ||
| DATE = "$(date +%Y%m%d-%H%M%S)" | ||
| JAVA_OPTS="-Dlog4j2.formatMsgNoLookups=true -Xmx8192m -XX:+UseG1GC -Xlog:gc*:${LOG_DIR}/fe.gc.log.$DATE:time" | ||
| http_port = 8030 | ||
| rpc_port = 9020 | ||
| query_port = 9030 | ||
| edit_log_port = 9010 | ||
| mysql_service_nio_enabled = true | ||
| sys_log_level = INFO | ||
| # config for meta and log | ||
| meta_dir = /opt/starrocks-meta | ||
| dump_log_dir = /opt/starrocks-log | ||
| sys_log_dir = /opt/starrocks-log | ||
| audit_log_dir = /opt/starrocks-log | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.