PowerStore

Enabling Replication In CSI PowerStore

Container Storage Modules (CSM) Replication sidecar is a helper container that is installed alongside a CSI driver to facilitate replication functionality. Such CSI drivers must implement dell-csi-extensions calls.

CSI driver for Dell PowerStore supports necessary extension calls from dell-csi-extensions. To be able to provision replicated volumes you would need to do the steps described in the following sections.

Before Installation

On Storage Array

Be sure to configure replication between multiple PowerStore instances using instructions provided by PowerStore storage.

You can ensure that you configured remote systems by navigating to the Protection tab and choosing Remote System in UI of your PowerStore instance.

You should see a list of remote systems with both Management State and Data Connection fields set to OK.

In Kubernetes

Ensure you installed CRDs and replication controller in your clusters.

To verify you have everything in order you can execute the following commands:

• Check controller pods:

  kubectl get pods -n dell-replication-controller
  

  Pods should be READY and RUNNING.
• Check that controller config map is properly populated:

  
  kubectl get cm -n dell-replication-controller dell-replication-controller-config -o yaml
  

  data field should be properly populated with cluster id of your choosing and, if using multi-cluster installation, your targets: parameter should be populated by list of target clusters IDs.

If you don’t have something installed or something is out-of-place, please refer to installation instructions here.

Installing Driver With Replication Module

To install the driver with replication enabled you need to ensure you have set Helm parameter controller.replication.enabled in your copy of example values.yaml file (usually called my-powerstore-settings.yaml, myvalues.yaml etc.).

Here is an example of what that would look like:

...
# controller: configure controller specific parameters
controller:
  ...
  # replication: allows to configure replication
  replication:
    enabled: true
    replicationContextPrefix: "powerstore"
    replicationPrefix: "replication.storage.dell.com"
...

You can leave other parameters like replicationContextPrefix, and replicationPrefix as they are.

After enabling the replication module you can continue to install the CSI driver for PowerStore following usual installation procedure, just ensure you’ve added necessary array connection information to secret.

NOTE: You need to install your driver on all clusters where you want to use replication. Both arrays must be accessible from each cluster.

Creating Storage Classes

To be able to provision replicated volumes you need to create properly configured storage classes on both source and target clusters.

A pair of storage classes on the source and target clusters would be essentially mirrored copies of one another. You can create them manually or with help from repctl.

Manual Storage Class Creation

You can find sample replication enabled storage class in the driver repository here.

It will look like this:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: "powerstore-replication"
provisioner: "csi-powerstore.dellemc.com"
reclaimPolicy: Delete
volumeBindingMode: Immediate
parameters:
  replication.storage.dell.com/isReplicationEnabled: "true"
  replication.storage.dell.com/mode: "ASYNC"
  replication.storage.dell.com/remoteStorageClassName: "powerstore-replication"
  replication.storage.dell.com/remoteClusterID: "tgt-cluster-id"
  replication.storage.dell.com/remoteSystem: "RT-0000"
  replication.storage.dell.com/rpo: Five_Minutes
  replication.storage.dell.com/ignoreNamespaces: "false"
  replication.storage.dell.com/volumeGroupPrefix: "csi"
  arrayID: "Unique"

Let’s go through each parameter and what it means:

• replication.storage.dell.com/isReplicationEnabled if set to true will mark this storage class as replication enabled, just leave it as true.
• replication.storage.dell.com/mode points to the replication mode you want to use. It should be one out of “ASYNC”, “SYNC” and “METRO”. The default value is ‘ASYNC’. If mode is set to ‘METRO’, the driver does not need remoteStorageClassName and remoteClusterID parameters as it supports METRO with single cluster configuration.

NOTE: Metro mode doesn’t involve the replication sidecar or the common replication controller, nor does it cause the creation of any replication related custom resources. It just needs the csi-powerstore driver.

• replication.storage.dell.com/remoteStorageClassName points to the name of the remote storage class. If you are using replication with the multi-cluster configuration you can make it the same as the current storage class name.
• replication.storage.dell.com/remoteClusterID represents ID of a remote Kubernetes cluster. It is the same ID you put in the replication controller config map.
• replication.storage.dell.com/remoteSystem is the name of the remote system as seen from the current PowerStore instance.
• replication.storage.dell.com/rpo is an acceptable amount of data, which is measured in units of time, that may be lost due to a failure. For ‘SYNC’ mode, RPO will be set to zero by default and not required to be specified. This parameter is not applicable for Metro mode.
• replication.storage.dell.com/ignoreNamespaces, if set to true the PowerStore driver will ignore the namespace in which the persistent volume claim resides and put every volume created using this storage class into a single volume group. This parameter is not applicable for Metro mode.
• replication.storage.dell.com/volumeGroupPrefix represents what string would be appended to the volume group name to differentiate them. It is important to not use the same prefix for different kubernetes clusters, otherwise any action on a replication group in one kubernetes cluster will impact the other. This parameter is not applicable for Metro mode.

NOTE: To configure the VolumeGroupPrefix, the name format of '<volumeGroupPrefix>-<namespace>-<Cluster Name>-<rpo>’ cannot be more than 63 characters.

• arrayID is a unique identifier of the storage array you specified in array connection secret.

Let’s follow up that with an example. Let’s assume you have two Kubernetes clusters and two PowerStore storage arrays:

• Clusters have IDs of cluster-1 and cluster-2
• Storage arrays connected between each other and show up as remote systems with names RT-0001 and RT-0002
• Cluster cluster-1 connected to array RT-0001
• Cluster cluster-2 connected to array RT-0002
• Storage array RT-0001 has a unique ID of PS000000001
• Storage array RT-0002 has a unique ID of PS000000002
• Replication mode is ASYNC

And this is what our pair of storage classes would look like:

StorageClass to be created in cluster-1:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: "powerstore-replication"
provisioner: "csi-powerstore.dellemc.com"
reclaimPolicy: Delete
volumeBindingMode: Immediate
parameters:
  replication.storage.dell.com/isReplicationEnabled: "true"
  replication.storage.dell.com/mode: "ASYNC"
  replication.storage.dell.com/remoteStorageClassName: "powerstore-replication"
  replication.storage.dell.com/remoteClusterID: "cluster-2"
  replication.storage.dell.com/remoteSystem: "RT-0002"
  replication.storage.dell.com/rpo: Five_Minutes
  replication.storage.dell.com/ignoreNamespaces: "false"
  replication.storage.dell.com/volumeGroupPrefix: "csi"
  arrayID: "PS000000001"

StorageClass to be created in cluster-2:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: "powerstore-replication"
provisioner: "csi-powerstore.dellemc.com"
reclaimPolicy: Delete
volumeBindingMode: Immediate
parameters:
  replication.storage.dell.com/isReplicationEnabled: "true"
  replication.storage.dell.com/mode: "ASYNC"
  replication.storage.dell.com/remoteStorageClassName: "powerstore-replication"
  replication.storage.dell.com/remoteClusterID: "cluster-1"
  replication.storage.dell.com/remoteSystem: "RT-0001"
  replication.storage.dell.com/rpo: Five_Minutes
  replication.storage.dell.com/ignoreNamespaces: "false"
  replication.storage.dell.com/volumeGroupPrefix: "csi"
  arrayID: "PS000000002"

After creating storage class YAML files, they must be applied to your Kubernetes clusters with kubectl.

Storage Class Creation With repctl

repctl can simplify storage class creation by creating a pair of mirrored storage classes in both clusters (using a single storage class configuration) in one command.

To create storage classes with repctl you need to fill the config with necessary information. You can find an example in here, copy it, and modify it to your needs.

If you open this example you can see similar fields and parameters to what was seen in manual storage class creation.

Let’s use the same example from manual installation and see what its repctl config file would look like:

sourceClusterID: "cluster-1"
targetClusterID: "cluster-2"
name: "powerstore-replication"
driver: "powerstore"
reclaimPolicy: "Delete"
replicationPrefix: "replication.storage.dell.com"
parameters:
  arrayID:
    source: "PS000000001"
    target: "PS000000002"
  remoteSystem:
    source: "RT-0002"
    target: "RT-0001"
  mode : "ASYNC"
  rpo: "Five_Minutes"
  ignoreNamespaces: "false"
  volumeGroupPrefix: "csi"

After preparing the config you can apply it to both clusters with repctl. Just make sure you’ve added your clusters to repctl via the add command before.

To create storage classes just run ./repctl create sc --from-config <config-file> and storage classes will be applied to both clusters.

After creating storage classes you can make sure they are in place by using ./repctl get storageclasses command.

Provisioning Replicated Volumes

After installing the driver and creating storage classes you are good to create volumes using the newly created storage classes.

On your source cluster, create a PersistentVolumeClaim using one of the replication enabled Storage Classes. The CSI PowerStore driver will create a volume on the array, add it to a VolumeGroup and configure replication using the parameters provided in the replication enabled Storage Class.

NOTE:
To add or remove volumes in an existing SYNC Replication Group in PowerStore, first pause the replication group, add or delete the volume, and then resume the replication group.
Use the following commands:
> - Pause the replication group: repctl --rg <rg-id> exec -a suspend > - Resume the replication group: repctl --rg <rg-id> exec -a resume

Supported Replication Actions

The CSI PowerStore driver supports the following list of replication actions:

• FAILOVER_REMOTE
• UNPLANNED_FAILOVER_LOCAL
• REPROTECT_LOCAL
• SUSPEND
• RESUME
• SYNC