Skip to content

Commit

Permalink
Update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
xiaogaozi committed Sep 27, 2024
1 parent 01e97bf commit 9411871
Show file tree
Hide file tree
Showing 6 changed files with 82 additions and 32 deletions.
49 changes: 49 additions & 0 deletions docs/en/administration/upgrade-csi-driver.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,55 @@ kubectl apply -f ./k8s.yaml

Dealing with exceptions like this, alongside with comparing and merging YAML files can be wearisome, that's why [install via Helm](../getting_started.md#helm) is much more recommended on a production environment.

### Migrate to Helm installation {#migrate-to-helm}

Helm installation requires filling in `values.yaml` first - all changes you have made to `k8s.yaml`, as long as they are within the scope of normal use, can find the corresponding configuration fields in `values.yaml`. What you need to do just sort out the current configuration and fill them in `values.yaml`. Of course, if you have not customized `k8s.yaml` (nor directly modified the configuration of the production environment), then the migration will be very simple. Just skip the grooming step and just follow the instructions below to uninstall and reinstall.

#### Sort out the configuration and fill in `values.yaml` {#sort-out-the-configuration-and-fill-in-values-yaml}

Before you start, you need to determine the CSI Driver version you are currently using. You can directly use the method at the beginning of this document to determine. The following takes the upgrade from v0.18.0 to v0.21.0 as an example to explain how to sort out the configuration line by line and fill in `values.yaml`.

1. Use a browser to access GitHub and open the diffs of the two versions. This process requires manually entering the link, pay attention to the version number at the end of the link, for example [https://github.com/juicedata/juicefs-csi-driver/compare/v0.18.0..v0.21.0](https://github.com/juicedata/juicefs-csi-driver/compare/v0.18.0..v0.21.0), found `k8s.yaml` in the file list. All `k8s.yaml` changes introduced by the version update will be displayed on the page. Keep this page, when sorting out the configuration later, if you are not sure which changes are your cluster's customized configuration and which are modifications brought about by the upgrade, you can refer to this page to judge;
1. Find the `k8s.yaml` used in the current online cluster installation, and rename its copy to `k8s-online.yaml`. This document will also use this name to refer to the current online installation file later in this document. It must be noted that the file must accurately reflect the "current online configuration". If your team has temporarily modified the online configuration (such as using `kubectl edit` to temporarily add environment variables and modify the image), you need to confirm these changes and append to `k8s-online.yaml`;
1. Install the new version (the link here takes v0.21.0 as an example) of the CSI Driver [`k8s.yaml`](https://github.com/juicedata/juicefs-csi-driver/blob/94d4f95a5d0f15a7a430ea31257d725306e90ca4/deploy/k8s.yaml) downloaded to the local, and compared with the online configuration, you can directly run `vimdiff k8s.yaml k8s-online.yaml`;
1. Compare the configuration files line by line to determine whether each configuration modification is brought about by the upgrade or customized by your team. Determine whether these customizations need to be retained, and then fill them in `values.yaml`. If you're not sure how to fill it out, you can usually find clues by carefully reading the annotated documentation in `values.yaml`.

We have the following suggestions for writing `values.yaml`:

If the default Mount Pod image is [overridden](../guide/custom-image.md#overwrite-mount-pod-image) in `k8s-online.yaml` (you can pass the `JUICEFS_EE_MOUNT_IMAGE` environment variable, or the `juicefs/mount-image` field of StorageClass ), and an older version of the Mount Pod image is specified, we encourage you to discard this configuration, let the cluster upgrade with the CSI Driver, and enable the new version of the Mount Pod image, which is equivalent to upgrading the JuiceFS client along with the CSI Driver upgrade.

Dynamic provisioning requires [create StorageClass](../guide/pv.md#create-storage-class), while in Helm Values, StorageClass and [volume credentials](../guide/pv.md#volume-credentials) are managed together. In order to avoid leaving sensitive information in `values.yaml`, we generally recommend manually managing the file system authentication information and StorageClass, and then disabling StorageClass in `values.yaml`:

```yaml title="values.yaml"
storageClasses:
- enabled: false
```
#### Uninstall and reinstall {#uninstall-and-reinstall}
If you use the default container mounting or sidecar mode, uninstalling the CSI Driver will not affect the current service (new PVs cannot be created or mounted during this period). Only [process mount mode](../introduction.md#by-process) will interrupt services due to uninstallation. If you are not using this mode, the migration process has no impact on the running PV and can be performed with confidence.
If your environment is an offline cluster and you cannot directly pull the image from the external network, you also need to [move the image](./offline.md) in advance.
Prepare the operation and maintenance commands that need to be run in advance, such as:
```shell
# Uninstall the CSI Driver
kubectl delete -f k8s-online.yaml

# Reinstall with Helm. The configuration of different clusters can be managed using different values.yaml files. For example values-dev.yaml, values-prod.yaml.
# The CSI Driver has no special requirements for the installation namespace. You can modify it as needed, such as jfs-system.
helm upgrade --install juicefs-csi-driver . -f values.yaml -n kube-system
```

Run these commands, and after reinstallation, immediately observe the startup status of each component of the CSI Driver:

```shell
kubectl -n kube-system get pods -l app.kubernetes.io/name=juicefs-csi-driver
```

Wait for all components to be started, and then simply create an application pod for verification. You can refer to [our demonstration](../guide/pv.md#static-provisioning).

## Upgrade CSI Driver (mount by process mode) {#mount-by-process-upgrade}

[Mount by process](../introduction.md#by-process) means that JuiceFS Client runs inside CSI Node Service Pod, under this mode, upgrading CSI Driver will inevitably interrupt existing mounts, use one of below methods to carry out the upgrade.
Expand Down
16 changes: 8 additions & 8 deletions docs/en/guide/configurations.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ globalConfig:
# Clean cache when mount pod exits
juicefs-clean-cache: "true"

# Define an environment variable for mount pod
# Define an environment variable for mount pod
- pvcSelector:
matchLabels:
...
Expand All @@ -119,7 +119,7 @@ globalConfig:
- name: DEMO_FAREWELL
value: "Such a sweet sorrow"

# Mount some volumes to mount pod
# Mount some volumes to mount pod
- pvcSelector:
matchLabels:
...
Expand All @@ -131,12 +131,12 @@ globalConfig:
persistentVolumeClaim:
claimName: block-pv

# Select by StorageClass
# Select by StorageClass
- pvcSelector:
matchStorageClassName: juicefs-sc
terminationGracePeriodSeconds: 60

# Select by PVC
# Select by PVC
- pvcSelector:
matchName: pvc-name
terminationGracePeriodSeconds: 60
Expand Down Expand Up @@ -223,7 +223,7 @@ stringData:

JuiceFS Enterprise Edition:

```yaml {13}
```yaml {11}
apiVersion: v1
kind: Secret
metadata:
Expand Down Expand Up @@ -355,7 +355,7 @@ There are two ways to mount subdirectory, one is through the `--subdir` mount op

If you'd like to share the same file system across different namespaces, use the same set of volume credentials (Secret) in the PV definition:

```yaml {10-12,24-26}
```yaml {9-11,22-24}
apiVersion: v1
kind: PersistentVolume
metadata:
Expand Down Expand Up @@ -449,7 +449,7 @@ helm upgrade juicefs-csi-driver juicefs/juicefs-csi-driver -n kube-system -f ./v

### kubectl

Helm is absolutely recommended since kubectl installation means a lot of complex manual edits. Please migrate to Helm installation as soon as possible.
If you use the kubectl installation method, enabling this feature requires manual editing of the CSI Controller, which is complicated. Therefore, it is recommended to [migrate to Helm installation method](../administration/upgrade-csi-driver.md#migrate-to-helm).

Manually edit CSI Controller:

Expand Down Expand Up @@ -530,7 +530,7 @@ kubectl annotate --overwrite node minikube myjfs.juicefs.com/cacheGroup=region-1

And then modify relevant fields in SC:

```yaml {11-13}
```yaml {12-14}
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
Expand Down
4 changes: 2 additions & 2 deletions docs/en/guide/pv.md
Original file line number Diff line number Diff line change
Expand Up @@ -407,7 +407,7 @@ Read [Usage](../introduction.md#usage) to learn about dynamic provisioning. Dyna

Create PVC and example pod:

```yaml {13}
```yaml {14}
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
Expand Down Expand Up @@ -464,7 +464,7 @@ Generic ephemeral volume works similar to dynamic provisioning, thus you'll need

Declare generic ephemeral volume directly in pod definition:

```yaml {19-30}
```yaml {20-31}
apiVersion: v1
kind: Pod
metadata:
Expand Down
13 changes: 6 additions & 7 deletions docs/zh_cn/administration/upgrade-csi-driver.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,18 +85,18 @@ kubectl apply -f ./k8s.yaml

Helm 安装需要先填写 `values.yaml`——你对 `k8s.yaml` 做过的所有改动,只要属于正常使用范畴,都能在 `values.yaml` 中找到对应的配置字段,你需要做的就是梳理当前配置,将他们填写到 `values.yaml`。当然了,如果你并没有对 `k8s.yaml` 进行定制(也没有对线上环境直接修改过配置),那么迁移会非常简单,直接跳过梳理步骤,直接按照下方的指示卸载重装即可。

#### 梳理配置、填写 `values.yaml`
#### 梳理配置、填写 `values.yaml` {#sort-out-the-configuration-and-fill-in-values-yaml}

开始之前,你需要确定当前所使用的 CSI 驱动版本,可以直接使用本文开头的方法进行判断。下方以 v0.18.0 升级到 v0.21.0 为例,讲解如何逐行梳理配置,填写 `values.yaml`

1. 用浏览器访问 GitHub,打开两个版本的 diff。这个过程需要手动输入链接,注意链接末尾的版本号,例如 `https://github.com/juicedata/juicefs-csi-driver/compare/v0.18.0..v0.21.0`,在文件列表中找到 `k8s.yaml`。在页面中会显示版本更新所引入的所有 `k8s.yaml` 变动。保留这个页面,稍后梳理配置的时候,如果不确定哪些变动是你的集群定制配置,哪些是升级所带来的修改,都可以参照这个页面来判断;
1. 用浏览器访问 GitHub,打开两个版本的 diff。这个过程需要手动输入链接,注意链接末尾的版本号,例如 [https://github.com/juicedata/juicefs-csi-driver/compare/v0.18.0..v0.21.0](https://github.com/juicedata/juicefs-csi-driver/compare/v0.18.0..v0.21.0),在文件列表中找到 `k8s.yaml`。在页面中会显示版本更新所引入的所有 `k8s.yaml` 变动。保留这个页面,稍后梳理配置的时候,如果不确定哪些变动是你的集群定制配置,哪些是升级所带来的修改,都可以参照这个页面来判断;
1. 找到当前线上集群安装时所使用的 `k8s.yaml`,将其拷贝重命名为 `k8s-online.yaml`,本文档后续也用该名称来指代当前线上的安装文件。一定要注意,该文件必须能准确反映出「当前线上的配置」,如果你的团队临时修改过线上配置(比如使用 `kubectl edit` 临时添加环境变量、修改镜像),你需要对这些改动加以确认,并追加到 `k8s-online.yaml`
1. 将新版(此处链接以 v0.21.0 为例)CSI 驱动的 [`k8s.yaml`](https://github.com/juicedata/juicefs-csi-driver/blob/94d4f95a5d0f15a7a430ea31257d725306e90ca4/deploy/k8s.yaml) 下载到本地,与线上配置进行对比,可以直接运行 `vimdiff k8s.yaml k8s-online.yaml`
1. 逐行对比配置文件,确定每一处配置修改是升级带来的,还是你的团队进行的定制。判断这些定制是否需要保留,然后填写到 `values.yaml`。如果不确定如何填写,仔细阅读 `values.yaml` 中的注释文档,通常就能找到线索。

我们对撰写 `values.yaml` 有如下建议:

如果 `k8s-online.yaml`[覆盖了默认的 Mount Pod 镜像](../guide/custom-image.md)(可以通过 `JUICEFS_EE_MOUNT_IMAGE` 环境变量,或者 StorageClass 的 `juicefs/mount-image` 字段),并且指定了一个更老版本的 Mount 镜像,我们鼓励你丢弃该配置,让集群随着 CSI 驱动升级,启用新版 Mount 镜像,相当于伴随着 CSI 驱动升级,JuiceSF 客户端一并升级。
如果 `k8s-online.yaml`[覆盖了默认的 Mount Pod 镜像](../guide/custom-image.md)(可以通过 `JUICEFS_EE_MOUNT_IMAGE` 环境变量,或者 StorageClass 的 `juicefs/mount-image` 字段),并且指定了一个更老版本的 Mount Pod 镜像,我们鼓励你丢弃该配置,让集群随着 CSI 驱动升级,启用新版 Mount Pod 镜像,相当于伴随着 CSI 驱动升级,JuiceFS 客户端一并升级。

动态配置需要[创建 StorageClass](../guide/pv.md#create-storage-class),而在 Helm Values 中,StorageClass 和[文件系统认证信息](../guide/pv.md#volume-credentials)是捆绑一起管理的,为了避免将敏感信息留在 `values.yaml`,我们一般建议手动管理文件系统认证信息和 StorageClass,然后将 `values.yaml` 中的 StorageClass 禁用:

Expand All @@ -105,7 +105,7 @@ storageClasses:
- enabled: false
```
#### 卸载重装
#### 卸载重装 {#uninstall-and-reinstall}
如果你使用默认的容器挂载,或者 Sidecar 模式,那么卸载 CSI 驱动是不影响当前服务的(期间新的 PV 无法创建、挂载)。只有[进程挂载模式](../introduction.md#by-process)会因为卸载而中断服务。如果你没有在使用该模式,迁移过程对正在运行的 PV 完全没有影响,可以放心执行。
Expand All @@ -117,9 +117,8 @@ storageClasses:
# 卸载当前 CSI 驱动
kubectl delete -f k8s-online.yaml

# 用 Helm 重装,不同集群的配置可以用不同的 values.yaml 文件来进行管理
# 比如 values-dev.yaml, values-prod.yaml
# CSI 驱动对安装命名空间没有特殊要求,你可以按需修改,比如 jfs-system
# 用 Helm 重装,不同集群的配置可以用不同的 values.yaml 文件来进行管理,比如 values-dev.yaml, values-prod.yaml。
# CSI 驱动对安装命名空间没有特殊要求,你可以按需修改,比如 jfs-system。
helm upgrade --install juicefs-csi-driver . -f values.yaml -n kube-system
```

Expand Down
28 changes: 15 additions & 13 deletions docs/zh_cn/guide/configurations.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ CSI 驱动的各种高级功能,以及使用 JuiceFS PV 的各项配置、CSI
由于 ConfigMap 功能强大、更加灵活,他将会或已经取代从前在 CSI 驱动中各种修改配置的方式,例如下方标有「不推荐」的小节,均为旧版中灵活性欠佳的实践,请及时弃用。**简而言之,如果一项配置已经在 ConfigMap 中得到支持,请优先在 ConfigMap 中对其进行配置,无需再使用旧版本中的实践。**

:::info 更新时效
修改 ConfigMap 以后,相关改动并不会立刻生效,这是由于挂载进容器的 ConfigMap 并非实时更新,而是定期同步(详见 [Kubernetes 官方文档](https://kubernetes.io/docs/concepts/configuration/configmap/#mounted-configmaps-are-updated-automatically))。
修改 ConfigMap 以后,相关改动并不会立刻生效,这是由于挂载进容器的 ConfigMap 并非实时更新,而是定期同步(详见 [Kubernetes 官方文档](https://kubernetes.io/zh-cn/docs/concepts/configuration/configmap/#%E8%A2%AB%E6%8C%82%E8%BD%BD%E7%9A%84-configmap-%E5%86%85%E5%AE%B9%E4%BC%9A%E8%A2%AB%E8%87%AA%E5%8A%A8%E6%9B%B4%E6%96%B0))。

如果希望立即生效,可以给 CSI 组件 pods 临时添加 annotation 来触发更新:

Expand Down Expand Up @@ -107,7 +107,7 @@ globalConfig:
# 退出时清理 cache
juicefs-clean-cache: "true"

# 为 mount pod 注入 env
# 为 mount pod 注入环境变量
- pvcSelector:
matchLabels:
...
Expand All @@ -117,7 +117,7 @@ globalConfig:
- name: DEMO_FAREWELL
value: "Such a sweet sorrow"

# 挂载 volumes 到 mount pod
# 挂载 volumes 到 mount pod
- pvcSelector:
matchLabels:
...
Expand All @@ -128,13 +128,13 @@ globalConfig:
- name: block-devices
persistentVolumeClaim:
claimName: block-pv
# 选择特定的 StorageClass

# 选择特定的 StorageClass
- pvcSelector:
matchStorageClassName: juicefs-sc
terminationGracePeriodSeconds: 60

# 选择特定的 PVC
# 选择特定的 PVC
- pvcSelector:
matchName: pvc-name
terminationGracePeriodSeconds: 60
Expand Down Expand Up @@ -221,7 +221,7 @@ stringData:

企业版:

```yaml {13}
```yaml {11}
apiVersion: v1
kind: Secret
metadata:
Expand Down Expand Up @@ -260,7 +260,7 @@ spec:

在 `StorageClass` 定义中调整挂载参数。如果需要为不同应用使用不同挂载参数,则需要创建多个 `StorageClass`,单独添加所需参数。

注意,StorageClass 仅仅是动态配置下用于创建 PV 的「模板」,也正因此,**在 StorageClass 中修改挂载配置,不影响已经创建的 PV。**如果你需要调整挂载配置,需要删除 PVC 重建,或者直接[在 PV 级别调整挂载配置](#static-mount-options)。
注意,StorageClass 仅仅是动态配置下用于创建 PV 的「模板」,也正因此,**在 StorageClass 中修改挂载配置,不影响已经创建的 PV**。如果你需要调整挂载配置,需要删除 PVC 重建,或者直接[在 PV 级别调整挂载配置](#static-mount-options)。

```yaml {6-7}
apiVersion: storage.k8s.io/v1
Expand Down Expand Up @@ -353,7 +353,7 @@ mountOptions:

如果想要在不同命名空间中共享同一个文件系统,只需要让不同 PV 使用相同的文件系统认证信息(Secret)即可:

```yaml {10-12,24-26}
```yaml {9-11,22-24}
apiVersion: v1
kind: PersistentVolume
metadata:
Expand Down Expand Up @@ -426,8 +426,8 @@ CSI 驱动提供两种方式进行 PV 初始化:

此特性默认关闭,需要手动启用。启用的方式就是为 CSI Controller 增添 `--provisioner=true` 启动参数,并且删去原本的 sidecar 容器,相当于让 CSI Controller 主进程自行监听资源变更,并执行相应的初始化操作。请根据 CSI Controller 的安装方式,按照下方步骤启用。

:::info
[进程挂载模式](../introduction.md#by-process)不支持该功能
:::tip
[进程挂载模式](../introduction.md#by-process)不支持高级 PV 初始化功能
:::

### Helm
Expand All @@ -447,7 +447,9 @@ helm upgrade juicefs-csi-driver juicefs/juicefs-csi-driver -n kube-system -f ./v

### kubectl

如果是 kubectl 安装方式,启用该功能需要手动编辑 CSI Controller,操作较为复杂,因此建议[迁移到 Helm 安装方式](../administration/upgrade-csi-driver.md#migrate-to-helm)。
如果使用 kubectl 安装方式,启用该功能需要手动编辑 CSI Controller,操作较为复杂,因此建议[迁移到 Helm 安装方式](../administration/upgrade-csi-driver.md#migrate-to-helm)。

手动修改 CSI Controller:

```shell
kubectl edit sts -n kube-system juicefs-csi-controller
Expand Down Expand Up @@ -526,7 +528,7 @@ kubectl annotate --overwrite node minikube myjfs.juicefs.com/cacheGroup=region-1

然后在 `StorageClass` 中修改相关配置:

```yaml {11-13}
```yaml {12-14}
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
Expand Down
4 changes: 2 additions & 2 deletions docs/zh_cn/guide/pv.md
Original file line number Diff line number Diff line change
Expand Up @@ -406,7 +406,7 @@ storageClasses:
创建 PVC 和应用 Pod,示范如下:
```yaml {13}
```yaml {14}
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
Expand Down Expand Up @@ -464,7 +464,7 @@ JuiceFS CSI 驱动的通用临时卷用法与「动态配置」类似,因此

在 Pod 定义中声明使用通用临时卷:

```yaml {19-30}
```yaml {20-31}
apiVersion: v1
kind: Pod
metadata:
Expand Down

0 comments on commit 9411871

Please sign in to comment.