v1.1.0

Deprecations:

• Scheduler extender and mutating pod webhook are no longer needed on any supported Kubernetes release because all of those support storage capacity tracking. The implementation got removed. The corresponding API (parameters, operator CRD) are still available, but don't have any effect.

Other changes:

• Kubernetes 1.22 to 1.25 are tested and supported. 1.26 should also work.
• Updated Go to 1.19.4.

v1.0.2

Other changes:

• fb0e4f6 XFS: fix creating volumes on OpenShift (issue #1058)
• 4ef304c operator-sdk 1.15.0, kustomize 4.4.1, controller-gen 0.7.0
• 86f3973 update to Go 1.17.6

v1.0.1

Bug fixes:

• 16dc613 pmem-csi: avoid false negative liveness probe (issue 1046)

Other changes:

• c8ed630 operator hub: explicitly list "Intel" as keyword
• 0cc7c13 docs: stop referencing Clear Linux
• 172c5b0 add section about OpenShift operator installation
• 5575aff add support for Kubernetes 1.22

v1.0.0

Major enhancements:

• The operator is now considered stable.
• OpenShift >= 4.6 is supported, including the schedule extensions. To avoid permission issues, the explicit and fixed UID was removed in favor of dynamic UID assignment on OpenShift.
• The controller can be deployed with multiple replicas to enhance availability of the scheduler extensions.
• ipmctl is included in the PMEM-CSI image. It can be run inside a container when the host OS doesn't provide it.
• The volume size of a volume provided by PMEM-CSI is now defined as the amount of storage that it consumes (PMEM or LVM blocks). This makes comparisons between free space and volume sizes more meaningful.
• Raw namespaces as exposed by vSphere can be converted through PMEM-CSI.
• Kubernetes 1.21 is supported, with storage capacity tracking enabled automatically. Generic ephemeral volumes should be used instead of CSI ephemeral volumes. The scheduler extensions are no longer needed on Kubernetes 1.21. The oldest tested and supported Kubernetes version is 1.19.

Bug fixes:

• Fixed failure to create namespaces on hardware configurations with additional alignment constraints.
• Kata Containers support works again.

Other changes:

• Eviction of the driver pods gets avoided by specifying resource requirements and priority.
• Better formatting of tables in the doc site, fixed broken links.
• Document troubleshooting steps.
• The log messages in the driver use structured key/value logging.
• Filesystem mount options support huge pages.
• Code and documentation cleanup.

v0.9.1

Other changes:

• f41366e deploy: give driver-registrar more time for requests
• 481a382 enhance resilience against LVM commands printing error messages
• 3502997 build: update to Go 1.16.1
• 94bffec deploy: set fsgroup for the pod in application example(s)
• 2d14673 and fcfda5f: wording changes in DEVELOPMENT.md

Known limitations:

• Raw block volumes are based on an "fsdax" namespace and thus only support dax through a filesystem. This is a technical limitation of PMEM and Kubernetes. A "devdax" namespace would support dax directly, but cannot be passed into a pod as a storage volume because it is a character device and Kubernetes expects a block device.
• When requesting volumes of a certain size in direct mode, the resulting volume may be smaller than requested.

v0.9.0

Major enhancements:

• Finalized the operator API, with PmemCSIDeployment as the name for the custom resource. The scheduler extensions can be enabled also in a deployment created by the operator. However, the Kubernetes scheduler still needs to be reconfigured manually. No more backward incompatible changes are planned.
• Replaced the custom controller<->node communication with distributed provisioning. This solves several race conditions that could have led to volume leaks. it also removes the need to create certificates when using only the CSI driver itself. Certificates are still needed for securing the communication between Kubernetes and the webhooks if that gets enabled.

Other changes:

• added support for Kubernetes 1.20
• 7a5b879: work around read-only /sys
• d42db56: faster volume deletion by overwriting with zero instead of random data
• 3e896ef: drop deprecated usage of v1beta1 mutation webhook api
• 0d36389: consistent object naming in deployments with app.kubernetes.io labels
• 17d4480: Output can be created in JSON format. All log output in the operator is "structured", i.e. message plus key/value pairs.
• updated dependencies and sidecars
• 262fd07: the default in the YAML files is to deploy in the pmem-csi namespace
• 3d63bf1: usage of NFD is recommended
• cafe248: fake device manager for testing without PMEM hardware

Known limitations:

• Raw block volumes are based on an "fsdax" namespace and thus only support dax through a filesystem. This is a technical limitation of PMEM and Kubernetes. A "devdax" namespace would support dax directly, but cannot be passed into a pod as a storage volume because it is a character device and Kubernetes expects a block device.

v0.8.1

Bug fixes in driver:

• c7ea8c9: fix rescheduling after CreateVolume failure
• 3d3a588: deploy: fix FSGroup support when using default FS

Other changes:

• some documentation fixes and enhancements
• smaller images because Python no longer gets installed
• fixed automatic creation of Operator Hub meta data files

Known limitations:

• Raw block volumes are based on an "fsdax" namespace and thus only support dax through a filesystem. This is a technical limitation of PMEM and Kubernetes. A "devdax" namespace would support dax directly, but cannot be passed into a pod as a storage volume because it is a character device and Kubernetes expects a block device.
• A race condition can lead to a volume leak: if deletion of a volume is triggered while the driver is currently starting up, then the PMEM-CSI controller may incorrectly assume that the volume was already deleted although it still exists on a node that the controller doesn't know about yet. The volume then gets removed in Kubernetes without removing it on that node.
• The operator API is still alpha and will continue to evolve.

v0.8.0

New features:

• The core features of PMEM-CSI are now production-ready. See the feature status section for details.
• Kubernetes 1.17, 1.18 and 1.19 are supported and tested.
• Added additional documentation for cluster administrators.
• Metrics support for Prometheus.
• memcached can be installed with PMEM volumes via KubeDB.
• The kubelet directory is configurable in the operator API.
• The operator will remove obsolete objects after an upgrade.
• Experimental deployment via YAML files with support for the CSIStorageCapacity alpha feature in Kubernetes 1.19.
• Usage example for the GenericEphemeralVolume feature in Kubernetes 1.19.

Breaking changes:

• Support for Kubernetes < 1.17 was removed because CSIDriver and CSINode v1 APIs are needed and only available starting with 1.17.

Other changes:

• Built with Go 1.15.2 and updated dependencies.
• Container images are now based on Debian instead of Clear Linux, with source code of packages under a copyleft license included in the image.
• Driver startup was simplified by merging the functionality of pmem-vg-init and pmem-ns-init into the pmem-csi-driver image.
• Various fixes and stability enhancements, better testing. In particular upgrade and downgrade between major releases is now covered.

Known limitations:

• Raw block volumes are based on an "fsdax" namespace and thus only support dax through a filesystem. This is a technical limitation of PMEM and Kubernetes. A "devdax" namespace would support dax directly, but cannot be passed into a pod as a storage volume because it is a character device and Kubernetes expects a block device.
• A race condition can lead to a volume leak: if deletion of a volume is triggered while the driver is currently starting up, then the PMEM-CSI controller may incorrectly assume that the volume was already deleted although it still exists on a node that the controller doesn't know about yet. The volume then gets removed in Kubernetes without removing it on that node.
• The operator API is still alpha and will continue to evolve.

v0.7.0

New features:

• installation via operator: simple installation without checking out the PMEM-CSI repository
• scheduler extensions: avoids scheduling of pods onto nodes with insufficient storage, in particular for ephemeral volumes
• Changing from direct mode to LVM and back works better: new volumes are created using the current mode and existing volumes using the old mode can be removed by the driver.
• Kata Containers are supported with full performance and DAX semantic when preparing volumes for it.
• A new example shows the deployment of memcached with PMEM volumes.
• An animated shell session demos how to install and use PMEM-CSI.
• Documentation is combined into a single document, available for each major release and the development branch.

Breaking changes:

• Support for Kubernetes 1.14 was removed because it lacks support for ephemeral inline volumes and is not supported officially by upstream anymore.
• Support for creating volumes using the "sector" namespace mode was removed.

Other changes:

• Various fixes and stability enhancements, better testing.

Known limitations:

• Raw block volumes are based on an "fsdax" namespace and thus only support dax through a filesystem. This is a technical limitation of PMEM and Kubernetes. A "devdax" namespace would support dax directly, but cannot be passed into a pod as a storage volume because it is a character device and Kubernetes expects a block device.

v0.6.0

New features:

• raw block volumes: applications can use volumes provisioned by PMEM-CSI as raw block devices. The advantage is that they have full control over how a filesystem is created on them.
• ephemeral inline volumes: ephemeral inline volumes are specified inside the pod spec and get created and destroyed together with the pod. This is ideal for local, non-persistent scratch space.
• PMEM-CSI in direct mode now allows admins to make changes (like creating new namespaces) while the driver is running.
• The sub-directories in deploy/kubernetes-* can be used as base for kustomize and/or kubectl kustomize.
• Source code was converted to Go modules, so it now also builds outside of GOPATH.

Breaking changes:

• Support for Kubernetes 1.13 was removed because its support for CSI is too limited.

Other changes:

• Enhanced documentation.
• Extended testing.

Deprecation warning:

• "Sector" namespace support is likely to be removed unless a need for it is found.

Known limitations:

• Raw block volumes are based on an "fsdax" namespace and thus only support dax through a filesystem. This is a technical limitation of PMEM and Kubernetes. A "devdax" namespace would support dax directly, but cannot be passed into a pod as a storage volume because it is a character device and Kubernetes expects a block device.
• Pod scheduling in Kubernetes is unaware of how much free PMEM exists on a node and thus a pod using an ephemeral inline volume or a persistent volume with WaitForFirstConsume volume binding mode may end up on a node where the volume cannot be created. In that case it is stuck until enough free PMEM becomes available.