PowerFlex

The CSI Driver for Dell PowerFlex can be deployed by using the provided Helm v3 charts and installation scripts on both Kubernetes and OpenShift platforms. For more detailed information on the installation scripts, review the script documentation.

Prerequisites

The following are requirements that must be met before installing the CSI Driver for Dell PowerFlex:

• Install Kubernetes or OpenShift (see supported versions)
• Install Helm 3.x
• Enable Zero Padding on PowerFlex
• Mount propagation is enabled on container runtime that is being used
• Install PowerFlex Storage Data Client
• If using Snapshot feature, satisfy all Volume Snapshot requirements
• A user must exist on the array with a role >= FrontEndConfigure
• If enabling CSM for Authorization, please refer to the Authorization deployment steps first
• If multipath is configured, ensure CSI-PowerFlex volumes are blacklisted by multipathd. See troubleshooting section for details
• Secure boot is not supported; ensure that secure boot is disabled in the BIOS.

Install Helm 3.x

Install Helm 3.x on the master node before you install the CSI Driver for Dell PowerFlex.

Steps

Run the command to install Helm 3.x.

curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

Enable Zero Padding on PowerFlex

Verify that zero padding is enabled on the PowerFlex storage pools that will be used. Use PowerFlex GUI or the PowerFlex CLI to check this setting. For more information to configure this setting, see Dell PowerFlex documentation.

Install PowerFlex Storage Data Client

The CSI Driver for PowerFlex requires you to have installed the PowerFlex Storage Data Client (SDC) on all Kubernetes nodes which run the node portion of the CSI driver. SDC could be installed automatically by CSI driver install on Kubernetes nodes with OS platform which support automatic SDC deployment; for Red Hat CoreOS (RHCOS) and RHEL. On Kubernetes nodes with OS version not supported by automatic install, you must perform the Manual SDC Deployment steps below. Refer to https://hub.docker.com/r/dellemc/sdc for supported OS versions. Please visit E-Lab Navigator for specific Dell Storage platform host operating system level support matrices.

NOTE: To install CSI driver for Powerflex with automated SDC deployment, you need below two packages on worker nodes.

1. libaio
2. numactl-libs

Optional: For a typical install, you will pull SDC kernel modules from the Dell FTP site, which is set up by default. Some users might want to mirror this repository to a local location. The PowerFlex KB article has instructions on how to do this.

Manual SDC Deployment

For detailed PowerFlex installation procedure, see the Dell PowerFlex Deployment Guide. Install the PowerFlex SDC as follows:

Steps

1. Download the PowerFlex SDC from Dell Online support. The filename is EMC-ScaleIO-sdc-*.rpm, where * is the SDC name corresponding to the PowerFlex installation version.
2. Export the shell variable MDM_IP in a comma-separated list using export MDM_IP=xx.xxx.xx.xx,xx.xxx.xx.xx, where xxx represents the actual IP address in your environment. This list contains the IP addresses of the MDMs.
3. Install the SDC per the Dell PowerFlex Deployment Guide:
   • For Red Hat Enterprise Linux and CentOS, run rpm -iv ./EMC-ScaleIO-sdc-*.x86_64.rpm, where * is the SDC name corresponding to the PowerFlex installation version.
4. To add more MDM_IP for multi-array support, run /opt/emc/scaleio/sdc/bin/drv_cfg --add_mdm --ip 10.xx.xx.xx.xx,10.xx.xx.xx

Installation Wizard prerequisite, secret update:

When the driver is installed using values generated by installation wizard, then the user needs to update the secret for driver by patching the MDM keys, as follows:

Steps

• echo -n '<MDM_IPS>' | base64
• kubectl patch secret vxflexos-config -n vxflexos -p "{\"data\": { \"MDM\": \"<GENERATED_BASE64>\"}}"

(Optional) Volume Snapshot Requirements

For detailed snapshot setup procedure, click here.

Install the Driver

Steps

1. Run git clone -b v2.11.0 https://github.com/dell/csi-powerflex.git to clone the git repository.

2. A namespace for the driver is expected prior to running the command below. If one is not created already, you can run kubectl create namespace vxflexos to create a new one. Note that the namespace can be any user-defined name that follows the conventions for namespaces outlined by Kubernetes. In this example we assume that the namespace is ‘vxflexos’

3. Collect information from the PowerFlex SDC by executing the get_vxflexos_info.sh script located in the scripts directory. This script shows the VxFlex OS system ID and MDM IP addresses. Make a note of the values for these parameters as they must be entered into samples/secret.yaml.

4. Prepare samples/secret.yaml for driver configuration. The following table lists driver configuration parameters for multiple storage arrays.

   Parameter	Description	Required	Default
   username	Username for accessing PowerFlex system. If authorization is enabled, username will be ignored.	true	-
   password	Password for accessing PowerFlex system. If authorization is enabled, password will be ignored.	true	-
   systemID	PowerFlex system name or ID.	true	-
   allSystemNames	List of previous names of powerflex array if used for PV create	false	-
   endpoint	REST API gateway HTTPS endpoint/PowerFlex Manager public IP for PowerFlex system. If authorization is enabled, endpoint should be the HTTPS localhost endpoint that the authorization sidecar will listen on	true	-
   skipCertificateValidation	Determines if the driver is going to validate certs while connecting to PowerFlex REST API interface.	true	true
   isDefault	An array having isDefault=true is for backward compatibility. This parameter should occur once in the list.	false	false
   mdm	mdm defines the MDM(s) that SDC should register with on start. This should be a list of MDM IP addresses or hostnames separated by comma.	true	-
   nasName	nasName defines what NAS should be used for NFS volumes. NFS volumes are supported on arrays version >=4.0.x	false	""

   Example: samples/secret.yaml

- username: "admin"
  password: "Password123"
  systemID: "2b11bb111111bb1b"
  endpoint: "https://127.0.0.2"
  skipCertificateValidation: true 
  isDefault: true 
  mdm: "10.0.0.3,10.0.0.4"

Example: samples/secret.yaml for PowerFlex storage system v4.0.x

- username: "admin"
  password: "Password123"
  systemID: "2b11bb111111bb1b"
  endpoint: "https://127.0.0.2"
  skipCertificateValidation: true
  isDefault: true
  mdm: "10.0.0.3,10.0.0.4"
  nasName : "nasServer"

NOTE: To use multiple arrays, copy and paste section above for each array. Make sure isDefault is set to true for only one array.

If replication feature is enabled, ensure the secret includes all the PowerFlex arrays involved in replication.

After editing the file, run the below command to create a secret called vxflexos-config. This assumes vxflexos is release name, but it can be modified during install:

kubectl create secret generic vxflexos-config -n vxflexos --from-file=config=samples/secret.yaml

Use the below command to replace or update the secret:

kubectl create secret generic vxflexos-config -n vxflexos --from-file=config=samples/secret.yaml -o yaml --dry-run=client | kubectl replace -f -

NOTE:

• It is mandatory to use SDC 4.5.2.1 for OpenShift 4.16 and above.
• The user needs to validate the YAML syntax and array-related key/values while replacing the vxflexos-creds secret.
• If you want to create a new array or update the MDM values in the secret, you will need to reinstall the driver. If you change other details, such as login information, the secret will dynamically update – see dynamic-array-configuration for more details.
• Old json format of the array configuration file is still supported in this release. If you already have your configuration in json format, you may continue to maintain it or you may transfer this configuration to yamlformat and replace/update the secret.
• “insecure” parameter has been changed to “skipCertificateValidation” as insecure is deprecated and will be removed from use in config.yaml or secret.yaml in a future release. Users can continue to use any one of “insecure” or “skipCertificateValidation” for now. The driver would return an error if both parameters are used.
• Please note that log configuration parameters from v1.5 will no longer work in v2.0 and higher. Please refer to the Dynamic Logging Configuration section in Features for more information.
• If the user is using complex K8s version like “v1.21.3-mirantis-1”, use this kubeVersion check in helm/csi-unity/Chart.yaml file. kubeVersion: “>= 1.21.0-0 < 1.29.0-0”

5. Default logging options are set during Helm install. To see possible configuration options, see the Dynamic Logging Configuration section in Features.

6. If using automated SDC deployment:

   • Check the SDC container image is the correct version for your version of PowerFlex.

7. Download the default values.yaml file

   cd dell-csi-helm-installer && wget -O myvalues.yaml https://github.com/dell/helm-charts/raw/csi-vxflexos-2.11.0/charts/csi-vxflexos/values.yaml
   

8. If you are using custom images, check the fields under images in my-vxflexos-settings.yaml to make sure that they are pointing to the correct image repository.

9. Look over all the other fields myvalues.yaml and fill in/adjust any as needed. All the fields are described here:

10. Install the driver using csi-install.sh bash script by running cd dell-csi-helm-installer && ./csi-install.sh --namespace vxflexos --values myvalues.yaml --helm-charts-version <version>. You may modify the release name with the --release arg. If arg is not provided, release will be named vxflexos by default. Alternatively, to do a helm install solely with Helm charts (without shell scripts), refer to helm/README.md.

NOTE:

• The parameter --helm-charts-version is optional and if you do not specify the flag, by default the csi-install.sh script will clone the version of the helm chart that is specified in the driver’s csi-install.sh file. If you wish to install the driver using a different version of the helm chart, you need to include this flag. Also, remember to delete the helm-charts repository present in the csi-powerflex directory if it was cloned before.

• For detailed instructions on how to run the install scripts, refer to the README.md in the dell-csi-helm-installer folder.

• Install script will validate MDM IP(s) in vxflexos-config secret and creates a new field consumed by the init container and sdc-monitor container

• This install script also runs the verify.sh script. You will be prompted to enter the credentials for each of the Kubernetes nodes. The verify.sh script needs the credentials to check if SDC has been configured on all nodes.

• It is mandatory to run install script after changes to MDM configuration in vxflexos-config secret. Refer dynamic-array-configuration

• If an extended Kubernetes version is being used (e.g. v1.21.3-mirantis-1) and is failing the version check in Helm even though it falls in the allowed range, then you must go into helm/csi-vxflexos/Chart.yaml and replace the standard kubeVersion check with the commented-out alternative. Please note that this will also allow the use of pre-release alpha and beta versions of Kubernetes, which is not supported.

• (Optional) Enable additional Mount Options - A user is able to specify additional mount options as needed for the driver.

  • Mount options are specified in storageclass yaml under mkfsFormatOption.
  • WARNING: Before utilizing mount options, you must first be fully aware of the potential impact and understand your environment’s requirements for the specified option.

Certificate validation for PowerFlex Gateway REST API calls

This topic provides details about setting up the certificate for the CSI Driver for Dell PowerFlex.

Before you begin

As part of the CSI driver installation, the CSI driver requires a secret with the name vxflexos-certs-0 to vxflexos-certs-n based on the “.Values.certSecretCount” parameter present in the namespace vxflexos.

This secret contains the X509 certificates of the CA which signed PowerFlex gateway SSL certificate in PEM format.

The CSI driver exposes an install parameter in secret.yaml, skipCertificateValidation, which determines if the driver performs client-side verification of the gateway certificates.

skipCertificateValidation parameter is set to true by default, and the driver does not verify the gateway certificates.

If skipCertificateValidation is set to false, then the secret vxflexos-certs-n must contain the CA certificate for the array gateway.

If this secret is an empty secret, then the validation of the certificate fails, and the driver fails to start.

If the gateway certificate is self-signed or if you are using an embedded gateway, then perform the following steps.

1. To fetch the certificate, run the following command.

   
   openssl s_client -showcerts -connect <Gateway IP:Port> </dev/null 2>/dev/null | openssl x509 -outform PEM > ca_cert_0.pem
   

   Example:

   
   openssl s_client -showcerts -connect 1.1.1.1:443 </dev/null 2>/dev/null | openssl x509 -outform PEM > ca_cert_0.pem
   

2. Run the following command to create the cert secret with index ‘0’:

   
   kubectl create secret generic vxflexos-certs-0 --from-file=cert-0=ca_cert_0.pem -n vxflexos
   

   Use the following command to replace the secret:

   
   kubectl create secret generic vxflexos-certs-0 -n vxflexos --from-file=cert-0=ca_cert_0.pem -o yaml --dry-run | kubectl replace -f -
   

3. Repeat step 1 and 2 to create multiple cert secrets with incremental index (example: vxflexos-certs-1, vxflexos-certs-2, etc)

Notes:

• “vxflexos” is the namespace for Helm-based installation but namespace can be user-defined in operator-based installation.
• User can add multiple certificates in the same secret. The certificate file should not exceed more than 1Mb due to Kubernetes secret size limitation.
• Whenever certSecretCount parameter changes in myvalues.yaml user needs to uninstall and install the driver.
• Updating vxflexos-certs-n secrets is a manual process, unlike vxflexos-config. Users have to re-install the driver in case of updating/adding the SSL certificates or changing the certSecretCount parameter.

Storage Classes

For CSI driver for PowerFlex version 1.4 and later, dell-csi-helm-installer does not create any storage classes as part of the driver installation. A wide set of annotated storage class manifests have been provided in the samples folder. Use these samples to create new storage classes to provision storage.

What happens to my existing storage classes?

Upgrading from an older version of the driver: The storage classes will be deleted if you upgrade the driver. If you wish to continue using those storage classes, you can patch them and apply the annotation “helm.sh/resource-policy”: keep before performing an upgrade.

Note: If you continue to use the old storage classes, you may not be able to take advantage of any new storage class parameter supported by the driver.

Steps to create storage class: There are samples storage class yaml files available under samples/storageclass. These can be copied and modified as needed.

1. Edit storageclass.yaml if you need ext4 filesystem, storageclass-xfs.yaml if you want xfs filesystem and storageclass-nfs.yaml if you need nfs filesystem
2. Replace <STORAGE_POOL> with the storage pool you have.
3. Replace <SYSTEM_ID> with the system ID you have. Note there are two appearances in the file.
4. Edit storageclass.kubernetes.io/is-default-class to true if you want to set it as default, otherwise false.
5. If using storageclass-nfs.yaml Replace "nas-server" with the NAS server’s name you have.
6. Save the file and create it by using kubectl create -f storageclass.yaml / kubectl create -f storageclass-xfs.yaml/ kubectl create -f storageclass-nfs.yaml

NOTE:

• At least one storage class is required for one array.
• If you uninstall the driver and reinstall it, you can still face errors if any update in the myvalues.yaml file leads to an update of the storage class(es):

    Error: cannot patch "<sc-name>" with kind StorageClass: StorageClass.storage.k8s.io "<sc-name>" is invalid: parameters: Forbidden: updates to parameters are forbidden

In case you want to make such updates, ensure to delete the existing storage classes using the kubectl delete storageclass command.
Deleting a storage class has no impact on a running Pod with mounted PVCs. You cannot provision new PVCs until at least one storage class is newly created.

Volume Snapshot Class

Starting CSI PowerFlex v1.5, dell-csi-helm-installer will not create any Volume Snapshot Class during the driver installation. There is a sample Volume Snapshot Class manifest present in the samples/ folder. Please use this sample to create a new Volume Snapshot Class to create Volume Snapshots.