Skip to content

Commit

Permalink
doc(1.8.0, 1.7, 1.6): online volume expansion for encrypted volumes
Browse files Browse the repository at this point in the history
longhorn/longhorn-9902

Signed-off-by: Raphanus Lo <[email protected]>
  • Loading branch information
COLDTURNIP committed Dec 19, 2024
1 parent 3d063d7 commit 853e007
Show file tree
Hide file tree
Showing 6 changed files with 64 additions and 13 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ For more information, see [cryptsetup(8)](https://man7.org/linux/man-pages/man8/
csi.storage.k8s.io/node-publish-secret-namespace: "longhorn-system"
csi.storage.k8s.io/node-stage-secret-name: "longhorn-crypto"
csi.storage.k8s.io/node-stage-secret-namespace: "longhorn-system"
csi.storage.k8s.io/node-expand-secret-name: "longhorn-crypto"
csi.storage.k8s.io/node-expand-secret-namespace: "longhorn-system"
```
- Example of a StorageClass with a volume-specific Secret:
Expand All @@ -85,6 +87,8 @@ For more information, see [cryptsetup(8)](https://man7.org/linux/man-pages/man8/
csi.storage.k8s.io/node-publish-secret-namespace: ${pvc.namespace}
csi.storage.k8s.io/node-stage-secret-name: ${pvc.name}
csi.storage.k8s.io/node-stage-secret-namespace: ${pvc.namespace}
csi.storage.k8s.io/node-expand-secret-name: ${pvc.name}
csi.storage.k8s.io/node-expand-secret-namespace: ${pvc.namespace}
```
# Using an Encrypted Volume
Expand All @@ -96,9 +100,14 @@ A newly-created PVC remains in the `Pending` state until the associated Secret i

# Filesystem Expansion

Longhorn supports [offline expansion](../../../nodes-and-volumes/volumes/expansion/#encrypted-volume) for encrypted volumes.
Longhorn supports [both online and offline expansion](../../../nodes-and-volumes/volumes/expansion/#encrypted-volume) for encrypted volumes.

StorageClass parameters are needed to enable online expansion:

- `csi.storage.k8s.io/node-expand-secret-name`
- `csi.storage.k8s.io/node-expand-secret-namespace`

# History
- Encryption of volumes in `Filesystem` mode available starting v1.2.0 ([#1859](https://github.com/longhorn/longhorn/issues/1859))

- Encryption of volumes in `Block` mode available starting v1.6.0 ([#4883](https://github.com/longhorn/longhorn/issues/4883))
- Encryption of volumes in `Block` mode available starting v1.6.0 ([#4883](https://github.com/longhorn/longhorn/issues/4883))
12 changes: 10 additions & 2 deletions content/docs/1.6.4/nodes-and-volumes/volumes/expansion.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,15 @@ If a volume is reverted to a snapshot with smaller size, the frontend of the vol
#### Encrypted volume
Due to [the upstream limitation](https://kubernetes.io/blog/2022/09/21/kubernetes-1-25-use-secrets-while-expanding-csi-volumes-on-node-alpha/), Longhorn cannot handle **online** expansion for encrypted volumes automatically unless you enable the feature gate `CSINodeExpandSecret`.
Longhorn supports online expansion, and it depends on the Kubernetes:
- Kubernetes support [authenticated CSI storage resizing](https://kubernetes.io/blog/2023/12/15/csi-node-expand-secret-support-ga/) natively from v1.29.
- From [Kubernetes v1.25 to v1.28](https://kubernetes.io/blog/2022/09/21/kubernetes-1-25-use-secrets-while-expanding-csi-volumes-on-node-alpha/), the feature gate `CSINodeExpandSecret` is required.
Online expansion for encrypted volume can be enabled by [specifying encryption parameters in the StorageClass](../../../advanced-resources/security/volume-encryption#setting-up-kubernetes-secrets-and-storageclasses):
- `csi.storage.k8s.io/node-expand-secret-name`
- `csi.storage.k8s.io/node-expand-secret-namespace`
If you cannot enable it but still prefer to do online expansion, you can:
1. Login the node host the encrypted volume is attached to.
Expand All @@ -134,7 +142,7 @@ Longhorn currently does not support fully automatic expansion of the filesystem
kubectl -n longhorn-system exec -it <the share manager pod> -- resize2fs /dev/longhorn/<volume name>
```
> **Important**:
> **Important**:
> Online expansion is possible only for `ext4` volumes. Attempts to manually expand `xfs` volumes with `xfs_growfs` may initially appear to be successful, but issues occur when the workload is scaled up and the volume is reattached. In particular, the pods become stuck in the `ContainerCreating` state, and the logs show an error message about attempts to mount the filesystem.
##### Offline
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ For more information, see [cryptsetup(8)](https://man7.org/linux/man-pages/man8/
csi.storage.k8s.io/node-publish-secret-namespace: "longhorn-system"
csi.storage.k8s.io/node-stage-secret-name: "longhorn-crypto"
csi.storage.k8s.io/node-stage-secret-namespace: "longhorn-system"
csi.storage.k8s.io/node-expand-secret-name: "longhorn-crypto"
csi.storage.k8s.io/node-expand-secret-namespace: "longhorn-system"
```
- Example of a StorageClass with a volume-specific Secret:
Expand All @@ -85,6 +87,8 @@ For more information, see [cryptsetup(8)](https://man7.org/linux/man-pages/man8/
csi.storage.k8s.io/node-publish-secret-namespace: ${pvc.namespace}
csi.storage.k8s.io/node-stage-secret-name: ${pvc.name}
csi.storage.k8s.io/node-stage-secret-namespace: ${pvc.namespace}
csi.storage.k8s.io/node-expand-secret-name: ${pvc.name}
csi.storage.k8s.io/node-expand-secret-namespace: ${pvc.namespace}
```
# Using an Encrypted Volume
Expand All @@ -96,9 +100,14 @@ A newly-created PVC remains in the `Pending` state until the associated Secret i

# Filesystem Expansion

Longhorn supports [offline expansion](../../../nodes-and-volumes/volumes/expansion/#encrypted-volume) for encrypted volumes.
Longhorn supports [both online and offline expansion](../../../nodes-and-volumes/volumes/expansion/#encrypted-volume) for encrypted volumes.

StorageClass parameters are needed to enable online expansion:

- `csi.storage.k8s.io/node-expand-secret-name`
- `csi.storage.k8s.io/node-expand-secret-namespace`

# History
- Encryption of volumes in `Filesystem` mode available starting v1.2.0 ([#1859](https://github.com/longhorn/longhorn/issues/1859))

- Encryption of volumes in `Block` mode available starting v1.6.0 ([#4883](https://github.com/longhorn/longhorn/issues/4883))
- Encryption of volumes in `Block` mode available starting v1.6.0 ([#4883](https://github.com/longhorn/longhorn/issues/4883))
12 changes: 10 additions & 2 deletions content/docs/1.7.3/nodes-and-volumes/volumes/expansion.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,15 @@ If a volume is reverted to a snapshot with smaller size, the frontend of the vol
#### Encrypted volume
Due to [the upstream limitation](https://kubernetes.io/blog/2022/09/21/kubernetes-1-25-use-secrets-while-expanding-csi-volumes-on-node-alpha/), Longhorn cannot handle **online** expansion for encrypted volumes automatically unless you enable the feature gate `CSINodeExpandSecret`.
Longhorn supports online expansion, and it depends on the Kubernetes:
- Kubernetes support [authenticated CSI storage resizing](https://kubernetes.io/blog/2023/12/15/csi-node-expand-secret-support-ga/) natively from v1.29.
- From [Kubernetes v1.25 to v1.28](https://kubernetes.io/blog/2022/09/21/kubernetes-1-25-use-secrets-while-expanding-csi-volumes-on-node-alpha/), the feature gate `CSINodeExpandSecret` is required.
Online expansion for encrypted volume can be enabled by [specifying encryption parameters in the StorageClass](../../../advanced-resources/security/volume-encryption#setting-up-kubernetes-secrets-and-storageclasses):
- `csi.storage.k8s.io/node-expand-secret-name`
- `csi.storage.k8s.io/node-expand-secret-namespace`
If you cannot enable it but still prefer to do online expansion, you can:
1. Login the node host the encrypted volume is attached to.
Expand All @@ -134,7 +142,7 @@ Longhorn currently does not support fully automatic expansion of the filesystem
kubectl -n longhorn-system exec -it <the share manager pod> -- resize2fs /dev/longhorn/<volume name>
```
> **Important**:
> **Important**:
> Online expansion is possible only for `ext4` volumes. Attempts to manually expand `xfs` volumes with `xfs_growfs` may initially appear to be successful, but issues occur when the workload is scaled up and the volume is reattached. In particular, the pods become stuck in the `ContainerCreating` state, and the logs show an error message about attempts to mount the filesystem.
##### Offline
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ For more information, see [cryptsetup(8)](https://man7.org/linux/man-pages/man8/
csi.storage.k8s.io/node-publish-secret-namespace: "longhorn-system"
csi.storage.k8s.io/node-stage-secret-name: "longhorn-crypto"
csi.storage.k8s.io/node-stage-secret-namespace: "longhorn-system"
csi.storage.k8s.io/node-expand-secret-name: "longhorn-crypto"
csi.storage.k8s.io/node-expand-secret-namespace: "longhorn-system"
```
- Example of a StorageClass with a volume-specific Secret:
Expand All @@ -85,6 +87,8 @@ For more information, see [cryptsetup(8)](https://man7.org/linux/man-pages/man8/
csi.storage.k8s.io/node-publish-secret-namespace: ${pvc.namespace}
csi.storage.k8s.io/node-stage-secret-name: ${pvc.name}
csi.storage.k8s.io/node-stage-secret-namespace: ${pvc.namespace}
csi.storage.k8s.io/node-expand-secret-name: ${pvc.name}
csi.storage.k8s.io/node-expand-secret-namespace: ${pvc.namespace}
```
# Using an Encrypted Volume
Expand All @@ -96,9 +100,14 @@ A newly-created PVC remains in the `Pending` state until the associated Secret i

# Filesystem Expansion

Longhorn supports [offline expansion](../../../nodes-and-volumes/volumes/expansion/#encrypted-volume) for encrypted volumes.
Longhorn supports [both online and offline expansion](../../../nodes-and-volumes/volumes/expansion/#encrypted-volume) for encrypted volumes.

StorageClass parameters are needed to enable online expansion:

- `csi.storage.k8s.io/node-expand-secret-name`
- `csi.storage.k8s.io/node-expand-secret-namespace`

# History
- Encryption of volumes in `Filesystem` mode available starting v1.2.0 ([#1859](https://github.com/longhorn/longhorn/issues/1859))

- Encryption of volumes in `Block` mode available starting v1.6.0 ([#4883](https://github.com/longhorn/longhorn/issues/4883))
- Encryption of volumes in `Block` mode available starting v1.6.0 ([#4883](https://github.com/longhorn/longhorn/issues/4883))
14 changes: 11 additions & 3 deletions content/docs/1.8.0/nodes-and-volumes/volumes/expansion.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,15 @@ If a volume is reverted to a snapshot with smaller size, the frontend of the vol
#### Encrypted volume
Due to [the upstream limitation](https://kubernetes.io/blog/2022/09/21/kubernetes-1-25-use-secrets-while-expanding-csi-volumes-on-node-alpha/), Longhorn cannot handle **online** expansion for encrypted volumes automatically unless you enable the feature gate `CSINodeExpandSecret`.
Longhorn supports online expansion, and it depends on the Kubernetes:
- Kubernetes support [authenticated CSI storage resizing](https://kubernetes.io/blog/2023/12/15/csi-node-expand-secret-support-ga/) natively from v1.29.
- From [Kubernetes v1.25 to v1.28](https://kubernetes.io/blog/2022/09/21/kubernetes-1-25-use-secrets-while-expanding-csi-volumes-on-node-alpha/), the feature gate `CSINodeExpandSecret` is required.
Online expansion for encrypted volume can be enabled by [specifying encryption parameters in the StorageClass](../../../advanced-resources/security/volume-encryption#setting-up-kubernetes-secrets-and-storageclasses):
- `csi.storage.k8s.io/node-expand-secret-name`
- `csi.storage.k8s.io/node-expand-secret-namespace`
If you cannot enable it but still prefer to do online expansion, you can:
1. Login the node host the encrypted volume is attached to.
Expand All @@ -131,9 +139,9 @@ From v1.8.0, Longhorn supports fully automatic online expansion of the filesyste
- CSI plugin
- Share Manager, which manages the NFS export
If you have upgraded from a previous version, the Share Manager pods (one for each RWX volume) are not upgraded automatically, to avoid disruption during the upgrade.
If you have upgraded from a previous version, the Share Manager pods (one for each RWX volume) are not upgraded automatically, to avoid disruption during the upgrade.
After growing the block device, the CSI layer sends a resize command to the Share Manager to grow the filesystem within the block device. With a down-rev share-manager, the command fails with an "unimplemented" error code and so no expansion happens. To get the right image before the expansion, the simplest thing is to force a restart of the pod. Identify the Share Manager pod of the RWX volume (typically named `share-manager-<volume name>`) and delete it:
After growing the block device, the CSI layer sends a resize command to the Share Manager to grow the filesystem within the block device. With a down-rev share-manager, the command fails with an "unimplemented" error code and so no expansion happens. To get the right image before the expansion, the simplest thing is to force a restart of the pod. Identify the Share Manager pod of the RWX volume (typically named `share-manager-<volume name>`) and delete it:
```shell
kubectl -n longhorn-system delete pod <the share manager pod>
Expand Down

0 comments on commit 853e007

Please sign in to comment.