Skip to content

Commit

Permalink
Docs: recommend to set non-preempting PriorityClass for Mount Pod by …
Browse files Browse the repository at this point in the history
…default (#695)

* Docs: recommend to set non-preempting PriorityClass for Mount Pod by default

* Update

* Fix lint
  • Loading branch information
xiaogaozi authored Jul 17, 2023
1 parent cdb1be7 commit 0e9f6ae
Show file tree
Hide file tree
Showing 6 changed files with 35 additions and 27 deletions.
6 changes: 5 additions & 1 deletion docs/en/administration/going-production.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,13 @@ Below settings are recommended for a production environment.

* [Configure more readable names for PV directory](../guide/pv.md#using-path-pattern)
* Enable [Automatic Mount Point Recovery](../guide/pv.md#automatic-mount-point-recovery)
* The `--writeback` option is strongly advised against, as it can easily cause data loss especially when used inside containers, if not properly managed. See ["Write Cache in Client (Community Edition)"](https://juicefs.com/docs/community/cache_management/#writeback) and ["Write Cache in Client (Cloud Service)"](https://juicefs.com/docs/cloud/guide/cache/#client-write-cache).
* The `--writeback` option is strongly advised against, as it can easily cause data loss especially when used inside containers, if not properly managed. See ["Write Cache in Client (Community Edition)"](/docs/community/cache_management/#writeback) and ["Write Cache in Client (Cloud Service)"](/docs/cloud/guide/cache/#client-write-cache).
* When cluster is low on resources, refer to optimization techniques in [Resource Optimization](../guide/resource-optimization.md).

## Mount Pod settings {#mount-pod-settings}

* It's recommended to set non-preempting PriorityClass for Mount Pod, see [documentation](../guide/resource-optimization.md#set-non-preempting-priorityclass-for-mount-pod) for details.

## Configure mount pod monitoring {#monitoring}

By default (not using `hostNetwork`), the mount pod provides a metrics API through port 9567 (you can also add [`metrics`](https://juicefs.com/docs/community/command_reference#mount) option in [`mountOptions`](../guide/pv.md#mount-options) to customize the port number), the port name is `metrics`, so the monitoring configuration of Prometheus can be configured as follows.
Expand Down
19 changes: 8 additions & 11 deletions docs/en/guide/pv.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,8 +299,6 @@ parameters:

Read [Usage](../introduction.md#usage) to learn about dynamic provisioning. Dynamic provisioning automatically creates PV for you, and the parameters needed by PV resides in StorageClass, thus you'll have to [create a StorageClass](#create-storage-class) in advance.

### Deploy

Create PVC and example pod:

```yaml {13}
Expand Down Expand Up @@ -540,7 +538,6 @@ Strictly speaking, dynamic provisioning doesn't inherently support mounting a ex
## Use more readable names for PV directory {#using-path-pattern}

:::tip

Not supported in [mount by process mode](../introduction.md#by-process).
:::

Expand Down Expand Up @@ -747,11 +744,11 @@ tmpfs 64M 0 64M 0% /dev
JuiceFS:ce-secret 100G 0 100G 0% /data-0
```

### PV expansion {#pv-resize}
### PV expansion {#pv-expansion}

In JuiceFS CSI Driver version 0.21.0 and above, PersistentVolume expansion is supported (only dynamic PersistentVolume is supported). You need to specify `allowVolumeExpansion: true` in StorageClass, and specify the Secret to be used when expanding the capacity, which mainly provides authentication information of the file system, for example:
In JuiceFS CSI Driver version 0.21.0 and above, PersistentVolume expansion is supported (only [dynamic provisioning](#dynamic-provisioning) is supported). You need to specify `allowVolumeExpansion: true` in [StorageClass](#create-storage-class), and specify the Secret to be used when expanding the capacity, which mainly provides authentication information of the file system, for example:

```yaml {8-10}
```yaml {9-11}
apiVersion: storage.k8s.io/v1
kind: StorageClass
...
Expand All @@ -760,14 +757,14 @@ parameters:
csi.storage.k8s.io/node-publish-secret-namespace: default
csi.storage.k8s.io/provisioner-secret-name: juicefs-secret
csi.storage.k8s.io/provisioner-secret-namespace: default
csi.storage.k8s.io/controller-expand-secret-name: juicefs-secret # same as provisioner-secret-name
csi.storage.k8s.io/controller-expand-secret-namespace: default # same as provisioner-secret-namespace
csi.storage.k8s.io/controller-expand-secret-name: juicefs-secret # same as provisioner-secret-name
csi.storage.k8s.io/controller-expand-secret-namespace: default # same as provisioner-secret-namespace
allowVolumeExpansion: true # indicates support for expansion
```

Expansion of the PersistentVolume can then be triggered by specifying a different (and larger) storage request by editing the PVC's spec field:
Expansion of the PersistentVolume can then be triggered by specifying a larger storage request by editing the PVC's `spec` field:

```yaml
```yaml {10}
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
Expand All @@ -777,7 +774,7 @@ spec:
- ReadWriteOnce
resources:
requests:
storage: 20Gi # specify new size here
storage: 20Gi # Specify a larger size here
```

### Access modes {#access-modes}
Expand Down
4 changes: 3 additions & 1 deletion docs/en/guide/resource-optimization.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,9 @@ storageClasses:
## Set non-preempting PriorityClass for Mount Pod {#set-non-preempting-priorityclass-for-mount-pod}

:::tip
If the mount mode of CSI Driver is ["Sidecar mode"](../introduction.md#sidecar), the following problems will not be encountered.

- It's recommended to set non-preempting PriorityClass for Mount Pod by default.
- If the mount mode of CSI Driver is ["Sidecar mode"](../introduction.md#sidecar), the following problems will not be encountered.
:::

When CSI Node creates a Mount Pod, it will set PriorityClass to `system-node-critical` by default, so that the Mount Pod will not be evicted when the node resources are insufficient.
Expand Down
6 changes: 5 additions & 1 deletion docs/zh_cn/administration/going-production.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,13 @@ sidebar_position: 1

* [配置更加易读的 PV 目录名称](../guide/pv.md#using-path-pattern)
* 启用[「挂载点自动恢复」](../guide/pv.md#automatic-mount-point-recovery)
* 不建议使用 `--writeback`,容器场景下,如果配置不当,极易引发丢数据等事故,详见[「客户端写缓存(社区版)」](https://juicefs.com/docs/zh/community/cache_management#writeback)[「客户端写缓存(云服务)」](https://juicefs.com/docs/zh/cloud/guide/cache/#client-write-cache)
* 不建议使用 `--writeback`,容器场景下,如果配置不当,极易引发丢数据等事故,详见[「客户端写缓存(社区版)」](/docs/zh/community/cache_management#writeback)[「客户端写缓存(云服务)」](/docs/zh/cloud/guide/cache/#client-write-cache)
* 如果资源吃紧,参照[「资源优化」](../guide/resource-optimization.md)以调优

## Mount Pod 设置 {#mount-pod-settings}

* 建议为 Mount Pod 设置非抢占式 PriorityClass,详见[文档](../guide/resource-optimization.md#set-non-preempting-priorityclass-for-mount-pod)

## 监控 Mount Pod {#monitoring}

默认设置下(未使用 `hostNetwork`),Mount Pod 通过 9567 端口提供监控 API(也可以通过在 [`mountOptions`](../guide/pv.md#mount-options) 中添加 [`metrics`](https://juicefs.com/docs/zh/community/command_reference#mount) 选项来自定义端口号),端口名为 `metrics`,因此可以按如下方式配置 Prometheus 的监控配置。
Expand Down
19 changes: 9 additions & 10 deletions docs/zh_cn/guide/pv.md
Original file line number Diff line number Diff line change
Expand Up @@ -388,7 +388,7 @@ spec:
storage: 1Gi
```
:::note
:::note 注意
在回收策略方面,临时卷与动态配置一致,因此如果将[默认 PV 回收策略](./resource-optimization.md#reclaim-policy)设置为 `Retain`,那么临时存储将不再是临时存储,PV 需要手动释放。
:::

Expand Down Expand Up @@ -537,8 +537,7 @@ spec:

## 配置更加易读的 PV 目录名称 {#using-path-pattern}

:::tip

:::tip 提示
[进程挂载模式](../introduction.md#by-process)不支持该功能。
:::

Expand Down Expand Up @@ -745,11 +744,11 @@ tmpfs 64M 0 64M 0% /dev
JuiceFS:ce-secret 100G 0 100G 0% /data-0
```

### PV 扩容 {#pv-resize}
### PV 扩容 {#pv-expansion}

在 JuiceFS CSI 驱动 0.21.0 及以上版本,支持 PersistentVolume 的扩容(仅支持动态 PersistentVolume)。需要在 StorageClass 中指定 `allowVolumeExpansion: true`,同时指定扩容时所需使用的 Secret,主要提供文件系统的认证信息,例如:
在 JuiceFS CSI 驱动 0.21.0 及以上版本,支持动态扩展 PersistentVolume 的容量(仅支持[动态配置](#dynamic-provisioning))。需要在 [StorageClass](#create-storage-class) 中指定 `allowVolumeExpansion: true`,同时指定扩容时所需使用的 Secret,主要提供文件系统的认证信息,例如:

```yaml {8-10}
```yaml {9-11}
apiVersion: storage.k8s.io/v1
kind: StorageClass
...
Expand All @@ -763,9 +762,9 @@ parameters:
allowVolumeExpansion: true # 表示支持扩容
```

然后通过编辑 PVC 的 spec 字段,指定不同的(和更大的)存储请求,可以触发 PersistentVolume 的扩充:
然后通过编辑 PVC 的 `spec` 字段,指定更大的存储请求,可以触发 PersistentVolume 的扩充:

```yaml
```yaml {10}
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
Expand All @@ -775,9 +774,9 @@ spec:
- ReadWriteOnce
resources:
requests:
storage: 20Gi # 在此处指定新的大小
storage: 20Gi # 在此处指定更大的容量
```

### 访问模式 {#access-modes}

JuiceFS PV 支持 `ReadWriteMany` 和 `ReadOnlyMany` 两种访问方式。根据使用 CSI 驱动的方式不同,在上方 PV/PVC(或 `volumeClaimTemplate`)定义中,填写需要的 `accessModes` 即可。
JuiceFS PV 支持 `ReadWriteMany` 和 `ReadOnlyMany` 两种访问方式。根据使用 CSI 驱动的方式不同,在上方 PVPVC(或 `volumeClaimTemplate`)定义中,填写需要的 `accessModes` 即可。
8 changes: 5 additions & 3 deletions docs/zh_cn/guide/resource-optimization.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ Kubernetes 的一大好处就是促进资源充分利用,在 JuiceFS CSI 驱

每一个使用着 JuiceFS PV 的容器,都对应着一个 Mount Pod(会智能匹配和复用),因此为 Mount Pod 配置合理的资源声明,将是最有效的优化资源占用的手段。关于配置资源请求(`request`)和约束(`limit`),可以详读 [Kubernetes 官方文档](https://kubernetes.io/zh-cn/docs/concepts/configuration/manage-resources-containers),此处不赘述。

JuiceFS Mount Pod 的 requests 默认为 1 CPU 和 1GiB Memory,limits 默认为 2 CPU 和 5GiB Memory。考虑到 JuiceFS 的使用场景多种多样,1U1G 的资源请求可能不一定适合你的集群,比方说:
JuiceFS Mount Pod 的 `requests` 默认为 1 CPU 和 1GiB Memory,`limits` 默认为 2 CPU 和 5GiB Memory。考虑到 JuiceFS 的使用场景多种多样,1C1G 的资源请求可能不一定适合你的集群,比方说:

* 实际场景下用量极低,比如 Mount Pod 只使用了 0.1 CPU、100MiB Memory,那么你应该尊重实际监控数据,将资源请求调整为 0.1 CPU,100MiB Memory,避免过大的 `requests` 造成资源闲置,甚至导致容器拒绝启动,或者抢占其他应用容器(Preemption)。对于 `limits`,你也可以根据实际监控数据,调整为一个大于 `requests` 的数值,允许突发瞬时的资源占用上升。
* 实际场景下用量更高,比方说 2 CPU、2GiB 内存,此时虽然 1U1G 的默认 `requests` 允许容器调度到节点上,但实际资源占用高于 `requests`,这便是「资源超售」(Overcommitment),严重的超售会影响集群稳定性,让节点出现各种资源挤占的问题,比如 CPU Throttle、OOM。因此这种情况下,你也应该根据实际用量,调整 `requests``limits`
* 实际场景下用量更高,比方说 2 CPU、2GiB 内存,此时虽然 1C1G 的默认 `requests` 允许容器调度到节点上,但实际资源占用高于 `requests`,这便是「资源超售」(Overcommitment),严重的超售会影响集群稳定性,让节点出现各种资源挤占的问题,比如 CPU Throttle、OOM。因此这种情况下,你也应该根据实际用量,调整 `requests``limits`

如果你安装了 [Kubernetes Metrics Server](https://github.com/kubernetes-sigs/metrics-server),可以方便地用类似下方命令查看 CSI 驱动组件的实际资源占用:

Expand Down Expand Up @@ -97,7 +97,9 @@ storageClasses:
## 为 Mount Pod 设置非抢占式 PriorityClass {#set-non-preempting-priorityclass-for-mount-pod}

:::tip 提示
如果 CSI 驱动的运行模式为[「Sidecar 模式」](../introduction.md#sidecar),则不会遇到以下问题。

- 建议默认为 Mount Pod 设置非抢占式 PriorityClass
- 如果 CSI 驱动的运行模式为[「Sidecar 模式」](../introduction.md#sidecar),则不会遇到以下问题。
:::

CSI Node 在创建 Mount Pod 时,会默认给其设置 PriorityClass 为 `system-node-critical`,目的是为了在机器资源不足时,Mount Pod 不会被驱逐。
Expand Down

0 comments on commit 0e9f6ae

Please sign in to comment.