PowerMax

The CSI Driver for Dell EMC PowerMax can be deployed by using the provided Helm v3 charts and installation scripts on both Kubernetes and OpenShift platforms. For more detailed information on the installation scripts, see the script documentation.

The controller section of the Helm chart installs the following components in a Deployment in the specified namespace:

  • CSI Driver for Dell EMC PowerMax
  • Kubernetes External Provisioner, which provisions the volumes
  • Kubernetes External Attacher, which attaches the volumes to the containers
  • Kubernetes External Snapshotter, which provides snapshot support
  • Kubernetes External Resizer, which resizes the volume
  • CSI PowerMax ReverseProxy (optional)

The node section of the Helm chart installs the following component in a DaemonSet in the specified namespace:

  • CSI Driver for Dell EMC PowerMax
  • Kubernetes Node Registrar, which handles the driver registration

Prerequisites

The following requirements must be met before installing the CSI Driver for Dell EMC PowerMax:

  • Install Kubernetes or OpenShift (see supported versions)
  • Install Helm 3
  • Fibre Channel requirements
  • iSCSI requirements
  • Certificate validation for Unisphere REST API calls
  • Mount propagation is enabled on container runtime that is being used
  • Linux multipathing requirements
  • If using Snapshot feature, satisfy all Volume Snapshot requirements

Install Helm 3

Install Helm 3 on the master node before you install the CSI Driver for Dell EMC PowerMax.

Steps

Run the curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash command to install Helm 3.

Fibre Channel Requirements

CSI Driver for Dell EMC PowerMax supports Fibre Channel communication. Ensure that the following requirements are met before you install the CSI Driver:

  • Zoning of the Host Bus Adapters (HBAs) to the Fibre Channel port director must be completed.
  • Ensure that the HBA WWNs (initiators) appear on the list of initiators that are logged into the array.
  • If the number of volumes that will be published to nodes is high, then configure the maximum number of LUNs for your HBAs on each node. See the appropriate HBA document to configure the maximum number of LUNs.

iSCSI Requirements

The CSI Driver for Dell EMC PowerMax supports iSCSI connectivity. These requirements are applicable for the nodes that use iSCSI initiator to connect to the PowerMax arrays.

Set up the iSCSI initiators as follows:

  • All Kubernetes nodes must have the iscsi-initiator-utils package installed.
  • Ensure that the iSCSI initiators are available on all the nodes where the driver node plugin will be installed.
  • Kubernetes nodes should have access (network connectivity) to an iSCSI director on the Dell EMC PowerMax array that has IP interfaces. Manually create IP routes for each node that connects to the Dell EMC PowerMax if required.
  • Ensure that the iSCSI initiators on the nodes are not a part of any existing Host (Initiator Group) on the Dell EMC PowerMax array.
  • The CSI Driver needs the port group names containing the required iSCSI director ports. These port groups must be set up on each Dell EMC PowerMax array. All the port group names supplied to the driver must exist on each Dell EMC PowerMax with the same name.

For more information about configuring iSCSI, you can refer Dell EMC Host Connectivity guide.

Certificate validation for Unisphere REST API calls

As part of the CSI driver installation, the CSI driver requires a secret with the name powermax-certs present in the namespace powermax. This secret contains the X509 certificates of the CA which signed the Unisphere SSL certificate in PEM format. This secret is mounted as a volume in the driver container. In earlier releases, if the install script did not find the secret, it created an empty secret with the same name. From the 1.2.0 release, the secret volume has been made optional. The install script no longer attempts to create an empty secret.

The CSI driver exposes an install parameter skipCertificateValidation which determines if the driver performs client-side verification of the Unisphere certificates. The skipCertificateValidation parameter is set to true by default, and the driver does not verify the Unisphere certificates.

If the skipCertificateValidation parameter is set to false and a previous installation attempt created an empty secret, then this secret must be deleted and re-created using the CA certs.

If the Unisphere certificate is self-signed or if you are using an embedded Unisphere, then perform the following steps:

  1. To fetch the certificate, run openssl s_client -showcerts -connect [Unisphere IP]:8443 </dev/null 2> /dev/null | openssl x509 -outform PEM > ca_cert.pem

    NOTE: The IP address varies for each user.

  2. To create the secret, run kubectl create secret generic powermax-certs --from-file=ca_cert.pem -n powermax

Ports in the port group

There are no restrictions around how many ports can be present in the iSCSI port groups provided to the driver.

The same applies to Fibre Channel where there are no restrictions on the number of FA directors a host HBA can be zoned to. See the best practices for host connectivity to Dell EMC PowerMax to ensure that you have multiple paths to your data volumes.

Linux multipathing requirements

CSI Driver for Dell EMC PowerMax supports Linux multipathing. Configure Linux multipathing before installing the CSI Driver.

Set up Linux multipathing as follows:

  • All the nodes must have the Device Mapper Multipathing package installed.
    NOTE: When this package is installed it creates a multipath configuration file which is located at /etc/multipath.conf. Please ensure that this file always exists.
  • Enable multipathing using mpathconf --enable --with_multipathd y
  • Enable user_friendly_names and find_multipaths in the multipath.conf file.

(Optional) Volume Snapshot Requirements

Applicable only if you decided to enable snapshot feature in values.yaml

snapshot:
  enabled: true

Volume Snapshot CRD’s

The Kubernetes Volume Snapshot CRDs can be obtained and installed from the external-snapshotter project on Github.

  • If on Kubernetes 1.19 (beta snapshots) use v3.0.x
  • If on Kubernetes 1.20/1.21 (v1 snapshots) use v4.0.x

Volume Snapshot Controller

The CSI external-snapshotter sidecar is split into two controllers to support beta Volume snapshots in Kubernetes 1.17 or later:

  • A common snapshot controller
  • A CSI external-snapshotter sidecar

The common snapshot controller must be installed only once in the cluster, irrespective of the number of CSI drivers installed in the cluster. On OpenShift clusters 4.4 and later, the common snapshot-controller is pre-installed. In the clusters where it is not present, it can be installed using kubectl and the manifests are available:

  • If on Kubernetes 1.19 (beta snapshots) use v3.0.x
  • If on Kubernetes 1.20 and 1.21 (v1 snapshots) use v4.0.x

NOTE:

Installation example

You can install CRDs and the default snapshot controller by running the following commands:

git clone https://github.com/kubernetes-csi/external-snapshotter/
cd ./external-snapshotter
git checkout release-<your-version>
kubectl create -f client/config/crd
kubectl create -f deploy/kubernetes/snapshot-controller

NOTE:

  • It is recommended to use 3.0.x version of snapshotter/snapshot-controller when using Kubernetes v1.19
  • When using Kubernetes 1.20/1.21 it is recommended to use 4.0.x version of snapshotter/snapshot-controller.
  • The CSI external-snapshotter sidecar is still installed along with the driver and does not involve any extra configuration.

Install the Driver

Steps

  1. Run git clone https://github.com/dell/csi-powermax.git to clone the git repository. This will include the Helm charts and dell-csi-helm-installer scripts.
  2. Ensure that you have created a namespace where you want to install the driver. You can run kubectl create namespace powermax to create a new one
  3. Edit the `helm/secret.yaml file, point to the correct namespace, and replace the values for the username and password parameters. These values can be obtained using base64 encoding as described in the following example:
    echo -n "myusername" | base64
    echo -n "mypassword" | base64
    

    where myusername and mypassword are credentials for a user with PowerMax privileges.

  4. Create the secret by running kubectl create -f helm/secret.yaml.
  5. If you are going to install the new CSI PowerMax ReverseProxy service, create a TLS secret with the name - csireverseproxy-tls-secret which holds an SSL certificate and the corresponding private key in the namespace where you are installing the driver.
  6. Copy the default values.yaml file cd helm && cp csi-powermax/values.yaml my-powermax-settings.yaml
  7. Edit the newly created file and provide values for the following parameters vi my-powermax-settings.yaml
Parameter Description Required Default
clusterPrefix Prefix that is used during the creation of various masking-related entities (Storage Groups, Masking Views, Hosts, and Volume Identifiers) on the array. The value that you specify here must be unique. Ensure that no other CSI PowerMax driver is managing the same arrays that are configured with the same prefix. The maximum length for this prefix is three characters. Yes “ABC”
controller Allows configuration of the controller-specific parameters. - -
node Allows configuration of the node-specific parameters. - -
tolerations Add tolerations as per requirement No -
nodeSelector Add node selectors as per requirement No -
defaultFsType Used to set the default FS type for external provisioner Yes ext4
portGroups List of comma-separated port group names. Any port group that is specified here must be present on all the arrays that the driver manages. For iSCSI Only “PortGroup1, PortGroup2, PortGroup3”
storageResourcePool This parameter must mention one of the SRPs on the PowerMax array that the symmetrixID specifies. This value is used to create the default storage class. Yes “SRP_1”
serviceLevel This parameter must mention one of the Service Levels on the PowerMax array. This value is used to create the default storage class. Yes “Bronze”
skipCertificateValidation Skip client-side TLS verification of Unisphere certificates No “True”
transportProtocol Set preferred transport protocol for the Kubernetes cluster which helps the driver choose between FC and iSCSI when a node has both FC and iSCSI connectivity to a PowerMax array. No Empty
nodeNameTemplate Used to specify a template that will be used by the driver to create Host/IG names on the PowerMax array. To use the default naming convention, leave this value empty. No Empty
global This section refers to configuration options for both CSI PowerMax Driver and Reverse Proxy - -
defaultCredentialsSecret This secret name refers to:
1. The Unisphere credentials if the driver is installed without proxy or with proxy in Linked mode.
2. The proxy credentials if the driver is installed with proxy in StandAlone mode.
3. The default Unisphere credentials if credentialsSecret is not specified for a management server.
Yes powermax-creds
storageArrays This section refers to the list of arrays managed by the driver and Reverse Proxy in StandAlone mode. - -
storageArrayId This refers to PowerMax Symmetrix ID. Yes 000000000001
endpoint This refers to the URL of the Unisphere server managing storageArrayId Yes if Reverse Proxy mode is StandAlone https://primary-1.unisphe.re:8443
backupEndpoint This refers to the URL of the backup Unisphere server managing storageArrayId, if Reverse Proxy is installed in StandAlone mode. No https://backup-1.unisphe.re:8443
managementServers This section refers to the list of configurations for Unisphere servers managing powermax arrays. - -
endpoint This refers to the URL of the Unisphere server Yes https://primary-1.unisphe.re:8443
credentialsSecret This refers to the user credentials for endpoint No primary-1-secret
skipCertificateValidation This parameter should be set to false if you want to do client-side TLS verification of Unisphere for PowerMax SSL certificates. No “True”
certSecret The name of the secret in the same namespace containing the CA certificates of the Unisphere server Yes, if skipCertificateValidation is set to false Empty
limits This refers to various limits for Reverse Proxy No -
maxActiveRead This refers to the maximum concurrent READ request handled by the reverse proxy. No 5
maxActiveWrite This refers to the maximum concurrent WRITE request handled by the reverse proxy. No 4
maxOutStandingRead This refers to maximum queued READ request when reverse proxy receives more than maxActiveRead requests. No 50
maxOutStandingWrite This refers to maximum queued WRITE request when reverse proxy receives more than maxActiveWrite requests. No 50
csireverseproxy This section refers to the configuration options for CSI PowerMax Reverse Proxy - -
enabled Boolean parameter which indicates if CSI PowerMax Reverse Proxy is going to be configured and installed.
NOTE: If not enabled, then there is no requirement to configure any of the following values.
No “False”
image This refers to the image of the CSI Powermax Reverse Proxy container. Yes dellemc/csipowermax-reverseproxy:v1.3.0
tlsSecret This refers to the TLS secret of the Reverse Proxy Server. Yes csirevproxy-tls-secret
deployAsSidecar If set to true, the Reverse Proxy is installed as a sidecar to the driver’s controller pod otherwise it is installed as a separate deployment. Yes “True”
port Specify the port number that is used by the NodePort service created by the CSI PowerMax Reverse Proxy installation Yes 2222
mode This refers to the installation mode of Reverse Proxy. It can be set to:
1. Linked: In this mode, the Reverse Proxy communicates with a primary or a backup Unisphere managing the same set of arrays.
2. StandAlone: In this mode, the Reverse Proxy communicates with multiple arrays managed by different Unispheres.
Yes “StandAlone”
  1. Install the driver using csi-install.sh bash script by running cd ../dell-csi-helm-installer && ./csi-install.sh --namespace powermax --values ../helm/my-powermax-settings.yaml

Note:

  • For detailed instructions on how to run the install scripts, see the readme document in the dell-csi-helm-installer folder.
  • There are a set of samples provided here to help you configure the driver with reverse proxy
  • This script also runs the verify.sh script in the same directory. You will be prompted to enter the credentials for each of the Kubernetes nodes. The verify.sh script needs the credentials to check if the iSCSI initiators have been configured on all nodes. You can also skip the verification step by specifying the --skip-verify-node option

Storage Classes

Starting CSI PowerMax v1.6, dell-csi-helm-installer will not create any storage classes as part of the driver installation. A wide set of annotated storage class manifests has been provided in the helm/samples/storageclass folder. Please use these samples to create new storage classes to provision storage. See this note for the driving reason behind this change.

What happens to my existing storage classes?

Upgrading from CSI PowerMax v1.5 driver The storage classes created as part of the installation have an annotation - “helm.sh/resource-policy”: keep set. This ensures that even after an uninstall or upgrade, the storage classes are not deleted. You can continue using these storage classes if you wish so.

Upgrading from an older version of the driver The storage classes will be deleted if you upgrade the driver. If you wish to continue using those storage classes, you can patch them and apply the annotation “helm.sh/resource-policy”: keep before performing an upgrade.

Note: If you continue to use the old storage classes, you may not be able to take advantage of any new storage class parameter supported by the driver.

Volume Snapshot Class

Starting CSI PowerMax v1.7, dell-csi-helm-installer will not create any Volume Snapshot Class during the driver installation. There is a sample Volume Snapshot Class manifest present in the helm/samples/ folder. Please use this sample to create a new Volume Snapshot Class to create Volume Snapshots.

Sample values file

The following sections have useful snippets from values.yaml file which provides more information on how to configure the CSI PowerMax driver along with CSI PowerMax ReverseProxy in various modes

CSI PowerMax driver without Proxy

In this mode, the CSI PowerMax driver can only connect to a single Unisphere server. So, you just specify a list of storage arrays and the address of the Unisphere server

global:
  defaultCredentialsSecret: powermax-creds
  storageArrays:
    - storageArrayId: "000000000001"
    - storageArrayId: "000000000002"
  managementServers:
    - endpoint: https://unisphere-address:8443

Note: If you provide multiple endpoints in the list of management servers, the installer will only use the first server in the list

CSI PowerMax driver with Proxy in Linked mode

In this mode, the CSI PowerMax ReverseProxy just acts as a passthrough for the RESTAPI calls and only provides limited functionality like rate limiting, backup Unisphere server. The CSI PowerMax driver is still responsible for the authentication with the Unisphere server.

The first endpoint in the list of management servers is the primary Unisphere server and if you provide a second endpoint, then it will be considered as the backup Unisphere’s endpoint.

global:
  defaultCredentialsSecret: powermax-creds
  storageArrays:
    - storageArrayId: "000000000001"
    - storageArrayId: "000000000002"
  managementServers:
    - endpoint: https://primary-unisphere:8443
      skipCertificateValidation: false
      certSecret: primary-cert
      limits:
        maxActiveRead: 5
        maxActiveWrite: 4
        maxOutStandingRead: 50
        maxOutStandingWrite: 50
    - endpoint: https://backup-unisphere:8443 #Optional

# "csireverseproxy" refers to the subchart csireverseproxy
csireverseproxy:
  # Set enabled to true if you want to use proxy
  enabled: true
  image: dellemc/csipowermax-reverseproxy:v1.3.0
  tlsSecret: csirevproxy-tls-secret
  deployAsSidecar: true
  port: 2222
  mode: Linked

Note: Since the driver is still responsible for authentication when used with Proxy in Linked mode, the credentials for both primary and backup Unisphere need to be the same.

CSI PowerMax driver with Proxy in StandAlone mode

This is the most advanced configuration which provides you the capability to connect to Multiple Unisphere servers. You can specify primary & backup Unisphere servers for each storage array. In case you have different credentials for your Unisphere servers, you can also specify different credential secrets.

global:
  defaultCredentialsSecret: powermax-creds
  storageArrays:
    - storageArrayId: "000000000001"
      endpoint: https://primary-1.unisphe.re:8443
      backupEndpoint: https://backup-1.unisphe.re:8443
    - storageArrayId: "000000000002"
      endpoint: https://primary-2.unisphe.re:8443
      backupEndpoint: https://backup-2.unisphe.re:8443
  managementServers:
    - endpoint: https://primary-1.unisphe.re:8443
      credentialsSecret: primary-1-secret
      skipCertificateValidation: false
      certSecret: primary-cert
      limits:
        maxActiveRead: 5
        maxActiveWrite: 4
        maxOutStandingRead: 50
        maxOutStandingWrite: 50
    - endpoint: https://backup-1.unisphe.re:8443
      credentialsSecret: backup-1-secret
      skipCertificateValidation: true
    - endpoint: https://primary-2.unisphe.re:8443
      credentialsSecret: primary-2-secret
      skipCertificateValidation: true
    - endpoint: https://backup-2.unisphe.re:8443
      credentialsSecret: backup-2-secret
      skipCertificateValidation: true

# "csireverseproxy" refers to the subchart csireverseproxy
csireverseproxy:
  # Set enabled to true if you want to use proxy
  enabled: true
  image: dellemc/csipowermax-reverseproxy:v1.3.0
  tlsSecret: csirevproxy-tls-secret
  deployAsSidecar: true
  port: 2222
  mode: StandAlone

Note: If the credential secret is missing from any management server details, the installer will try to use the defaultCredentialsSecret