Prerequisite

NVMe/TCP Requirements

1. Complete the NVMe network configuration to connect the hosts with the PowerFlex Storage array. Please refer Host Connectivity Guide for the best practices for attaching the hosts to a PowerFlex storage array.

2. Verify the initiators of each host are logged in to the PowerFlex Storage Array. CSM will perform the Host Registration of each host with the PowerFlex Array.

3. To ensure successful integration of NVMe protocols with the CSI Driver, the following conditions must be met:

   • Each OpenShift node that connects to Dell storage arrays must have a unique NVMe Qualified Name (NQN).
   • By default, the OpenShift deployment process for CoreOS assigns the same host NQN to all nodes. This value is stored in the file: /etc/nvme/hostnqn.
   • To resolve this and guarantee unique host NQNs across nodes, you can apply a machine configuration to your OpenShift Container Platform (OCP) cluster. One recommended approach is to add the following machine config:
   
   

   cat <<EOF > 99-worker-custom-nvme-hostnqn.yaml
   apiVersion: machineconfiguration.openshift.io/v1
   kind: MachineConfig
   metadata:
     labels:
       machineconfiguration.openshift.io/role: worker
     name: 99-worker-custom-nvme-hostnqn
   spec:
     config:
       ignition:
         version: 3.4.0
       systemd:
         units:
           - contents: |
               [Unit]
               Description=Custom CoreOS Generate NVMe Hostnqn
               [Service]
               Type=oneshot
               ExecStart=/usr/bin/sh -c '/usr/sbin/nvme gen-hostnqn > /etc/nvme/hostnqn'
               RemainAfterExit=yes
               [Install]
               WantedBy=multi-user.target
             enabled: true
             name: custom-coreos-generate-nvme-hostnqn.service
   EOF
   

4. Configure IO policy for native NVMe multipathing

   Use this comment to create the machine configuration to configure the native NVMe multipathing IO Policy to round robin.

   oc apply -f 99-workers-multipath-round-robin.yaml
   

   
   

   cat <<EOF> 71-nvmf-iopolicy-dell.rules
   ACTION=="add", SUBSYSTEM=="nvme-subsystem", ATTR{model}=="dellemc-PowerFlex",ATTR{iopolicy}="round-robin"
   EOF
   

   
   

   Example:

   cat <<EOF> 99-workers-multipath-round-robin.yaml
   apiVersion: machineconfiguration.openshift.io/v1
   kind: MachineConfig
   metadata:
     name: 99-workers-multipath-round-robin
     labels:
       machineconfiguration.openshift.io/role: worker
   spec:
     config:
       ignition:
         version: 3.4.0
       storage:
         files:
         - contents:
             source: data:text/plain;charset=utf-8;base64,$(cat 71-nvmf-iopolicy-dell.rules | base64 -w0)
             verification: {}
           filesystem: root
           mode: 420
           path: /etc/udev/rules.d/71-nvme-io-policy.rules 
   EOF
   

5. Configure NVMe reconnecting forever

   Use this command to create the machine configuration to configure the NVMe reconnect

   oc apply -f 99-workers-nvmf-ctrl-loss-tmo.yaml 
   

   
   

   cat <<EOF> 72-nvmf-ctrl_loss_tmo.rules
   ACTION=="add|change", SUBSYSTEM=="nvme", KERNEL=="nvme*", ATTR{ctrl_loss_tmo}="-1"
   EOF
   

   
   

   cat <<EOF> 99-workers-nvmf-ctrl-loss-tmo.yaml
   apiVersion: machineconfiguration.openshift.io/v1
   kind: MachineConfig
   metadata:
     name: 99-workers-nvmf-ctrl-loss-tmo
     labels:
       machineconfiguration.openshift.io/role: worker
   spec:
     config:
       ignition:
         version: 3.4.0
       storage:
         files:
         - contents:
             source: data:text/plain;charset=utf-8;base64,$(cat 72-nvmf-ctrl_loss_tmo.rules | base64 -w0)
             verification: {}
           filesystem: root
           mode: 420
           path: /etc/udev/rules.d/72-nvmf-ctrl_loss_tmo.rules
   EOF
   

Cluster requirements

All OpenShift nodes connecting to Dell storage arrays must use unique host NVMe Qualified Names (NQNs).

The OpenShift deployment process for CoreOS will set the same host NQN for all nodes. The host NQN is stored in the file /etc/nvme/hostnqn. One possible solution to ensure unique host NQNs is to add the following machine config to your OCP cluster:

cat <<EOF> 99-worker-custom-nvme-hostnqn.yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: 99-worker-custom-nvme-hostnqn
spec:
  config:
    ignition:
      version: 3.4.0
    systemd:
      units:
        - contents: |
            [Unit]
            Description=Custom CoreOS Generate NVMe Hostnqn

            [Service]
            Type=oneshot
            ExecStart=/usr/bin/sh -c '/usr/sbin/nvme gen-hostnqn > /etc/nvme/hostnqn'
            RemainAfterExit=yes

            [Install]
            WantedBy=multi-user.target
          enabled: true
          name: custom-coreos-generate-nvme-hostnqn.service
EOF

NFS Requirements

• Ensure that your nodes support mounting NFS volumes if using NFS.