Release Notes

Notifications

General:

  • Support via Slack is discontinued. For support, please reach out via the Dell Support Portal.
  • Container Storage Modules Volume Group Snapshotter is no longer supported.
  • Docker Hub images are discontinued. All deployments will be using images from quay.io.

Deprecation:

  • As of Container Storage Modules v1.16, the CSI PowerMax Reverse Proxy ConfigMap will not be supported. Please migrate to CSI PowerMax secret using the following guides Helm and Operator.
  • As of Container Storage Modules v1.16, the CSI PowerMax array ConfigMap will not be supported. Use the PowerMax sample to add values to the ConfigMap parameters and migrate to the latest ConfigMap using the following guide Operator.
  • Starting from Container Storage Modules 1.17.0, cert-csi tool is deprecated and will no longer receive updates or maintenance. Users are encouraged to adopt industry standard frameworks such as Kubernetes E2E , testing guidelines and OpenShift E2E for CSI driver validation.
  • The Container Storage Modules Application Mobility component has been deprecated and will no longer receive updates or maintenance.
  • Starting from CSM version 1.15, the Karavi topology component will no longer operate as a standalone service. Instead, topology data will be directly exported to the OpenTelemetry Collector

Release Notes for v1.15.0

New Features/Changes

Fixed Issues

Known Issues

Issue Workaround
Installing CSM Operator version 1.10.0 from the OpenShift Web Console may result in the new Operator instance becoming stuck in the upgradePending state. This prevents successful deployment.
Uninstall the affected Operator instance and reinstall it to restore normal operation.
When CSM Operator creates a deployment that includes secrets (e.g., observability, cert-manager, velero), these secrets are not deleted on uninstall and will be left behind. For example, the karavi-topology-tls, otel-collector-tls, and cert-manager-webhook-ca secrets will not be deleted.
This should not cause any issues on the system, but all secrets present on the cluster can be found with kubectl get secrets -A, and any unwanted secrets can be deleted with kubectl delete secret -n <secret-namespace> <secret-name>
In certain environments, users have encountered difficulties in installing drivers using the CSM Operator due to the ‘OOM Killed’ issue. This issue is attributed to the default resource requests and limits configured in the CSM Operator, which fail to meet the resource requirements of the user environments. OOM error occurs when a process in the container tries to consume more memory than the limit specified in resource configuration.
Before deploying the CSM Operator, it is crucial to adjust the memory and CPU requests and limits in the files config/manager.yaml, deploy/operator.yaml to align with the user’s environment requirements. If the containers running on the pod exceed the specified CPU and memory limits, the pod may get evicted. Currently CSM Operator does not support updating this configuration dynamically. CSM Operator needs to be redeployed for these updates to take effect in case it is already installed. Steps to manually update the resource configuration and then redeploy CSM Operator are available here
sdc:3.6.x/3.6.x.x is causing issues while installing the csi-powerflex driver on ubuntu/RHEL.
Workaround:
Change the powerflexSdc to sdc:3.6 in values.yaml
If the volume limit is exhausted and there are pending pods and PVCs due to exceed max volume count, the pending PVCs will be bound to PVs and the pending pods will be scheduled to nodes when the driver pods are restarted.
It is advised not to have any pending pods or PVCs once the volume limit per node is exhausted on a CSI Driver. There is an open issue reported with kubernetes at https://github.com/kubernetes/kubernetes/issues/95911 with the same behavior.
Resource quotas may not work properly with the CSI PowerFlex driver. PowerFlex is only able to assign storage in 8Gi chunks, so if a create volume call is made with a size not divisible by 8Gi, CSI-PowerFlex will round up to the next 8Gi boundary when it provisions storage – however, the resource quota will not record this size but rather the original size in the create request. This means that, for example, if a 10Gi resource quota is set, and a user provisions 10 1Gi PVCs, 80Gi of storage will actually be allocated, which is well over the amount specified in the resource quota.
For now, users should only provision volumes in 8Gi-divisible chunks if they want to use resource quotas.
Unable to update Host: A problem occurred modifying the host resource
This issue occurs when the nodes do not have unique hostnames or when an IP address/FQDN with same sub-domains are used as hostnames. The workaround is to use unique hostnames or FQDN with unique sub-domains
Automatic SRDF group creation is failing with “Unable to get Remote Port on SAN for Auto SRDF” for PowerMaxOS 10.1 arrays
Create the SRDF Group and add it to the storage class
When the driver is installed using CSM Operator , a few times, pods created using block volume are getting stuck in containercreating/terminating state or devices are not available inside the pod.
Update the daemonset with parameter mountPropagation: "Bidirectional" for volumedevices-path under volumeMounts section.
When running CSI-PowerMax with Replication in a multi-cluster configuration, the driver on the target cluster fails and the following error is seen in logs: error="CSI reverseproxy service host or port not found, CSI reverseproxy not installed properly"
The reverseproxy service needs to be created manually on the target cluster.
Storage capacity tracking does not return MaximumVolumeSize parameter. PowerScale is purely NFS based meaning it has no actual volumes. Therefore MaximumVolumeSize cannot be implemented if there is no volume creation.
CSI PowerScale 2.9.1 is compliant with CSI 1.6 specification since the field MaximumVolumeSize is optional.
If some older NFS exports /terminated worker nodes still in NFS export client list, CSI driver tries to add a new worker node it fails (For RWX volume).
User need to manually clean the export client list from old entries to make successful addition of new worker nodes.
fsGroupPolicy may not work as expected without root privileges for NFS only
https://github.com/kubernetes/examples/issues/260
To get the desired behavior set “RootClientEnabled” = “true” in the storage class parameter
PowerScale 9.5.0, Driver installation fails with session based auth, “HTTP/1.1 401 Unauthorized”
Fix is available in PowerScale >= 9.5.0.4
If the NVMeFC pod is not getting created and the host loses the ssh connection, causing the driver pods to go to error state
remove the nvme_tcp module from the host in case of NVMeFC connection
When a node goes down, the block volumes attached to the node cannot be attached to another node
This is a known issue and has been reported at https://github.com/kubernetes-csi/external-attacher/issues/215. Workaround:
1. Force delete the pod running on the node that went down
2. Delete the volumeattachment to the node that went down. Now the volume can be attached to the new node.
When driver node pods enter CrashLoopBackOff and PVC remains in pending state with one of the following events:
1. failed to provision volume with StorageClass <storage-class-name>: error generating accessibility requirements: no available topology found
2. waiting for a volume to be created, either by external provisioner “csi-powerstore.dellemc.com” or manually created by system administrator.
Check whether all array details present in the secret file are valid and remove any invalid entries if present.
Redeploy the driver.
In OpenShift 4.13, the root user is not allowed to perform write operations on NFS shares, when root squashing is enabled.
The workaround for this issue is to disable root squashing by setting allowRoot: “true” in the NFS storage class.
If the volume limit is exhausted and there are pending pods and PVCs due to exceed max volume count, the pending PVCs will be bound to PVs, and the pending pods will be scheduled to nodes when the driver pods are restarted.
It is advised not to have any pending pods or PVCs once the volume limit per node is exhausted on a CSI Driver. There is an open issue reported with Kubernetes at https://github.com/kubernetes/kubernetes/issues/95911 with the same behavior.
If two separate networks are configured for ISCSI and NVMeTCP, the driver may encounter difficulty identifying the second network (e.g., NVMeTCP).
This is a known issue, and the workaround involves creating a single network on the array to serve both ISCSI and NVMeTCP purposes.
When a PV/PVC is deleted in Kubernetes, it will trigger the deletion of the underlying volume and snapshot on the array as a default behaviour. This can result in a situation where the VolumeSnapshot and VolumeSnapshotContent will still show “readyToUse: true”, but leaves them unusable because it is no longer backed by underlying storage snapshot. This will not allow the creation of a PVC from snapshot and this could also lead to a data loss situations.
This is a known issue, and the workaround is use of retain policy on the various PV, VolumeSnapshot and VolumeSnapshotContent that you wish to use for cloning.
Nodes not getting registered on Unity XT.
Creating wrapper around hostname command inside the node pod’s driver container fails when -I flag is used. This will trigger fallback behaviour in driver and should fix the issue.
Topology-related node labels are not removed automatically.
Currently, when the driver is uninstalled, topology-related node labels are not getting removed automatically. There is an open issue in the Kubernetes to fix this. Until the fix is released, remove the labels manually after the driver un-installation using command kubectl label node <node_name> - - … Example: kubectl label node csi-unity.dellemc.com/array123-iscsi- Note: there must be - at the end of each label to remove it.
NFS Clone - Resize of the snapshot is not supported by Unity XT Platform, however, the user should never try to resize the cloned NFS volume.
Currently, when the driver takes a clone of NFS volume, it succeeds but if the user tries to resize the NFS volumesnapshot, the driver will throw an error.
A CSI ephemeral pod may not get created starting OpenShift 4.13 and fail with the error "error when creating pod: the pod uses an inline volume provided by CSIDriver csi-unity.dellemc.com, and the namespace has a pod security enforcement level that is lower than privileged."
This issue occurs because OpenShift 4.13 introduced the CSI Volume Admission plugin to restrict the use of a CSI driver capable of provisioning CSI ephemeral volumes during pod admission. Therefore, an additional label security.openshift.io/csi-ephemeral-volume-profile in csidriver.yaml file with the required security profile value should be provided.
Controller publish is taking too long to complete/ Health monitoring is causing Unity array to panic by opening multiple sessions/ There are error messages in the log context deadline exceeded, when health monitoring is enabled
Disable volume health monitoring on the node and keep it only at the controller level.