Observability
Container Storage Modules for Observability is part of the open-source suite of Kubernetes storage enablers for Dell products.
It is an OpenTelemetry agent that collects array-level metrics for Dell storage so they can be exported into a Prometheus database. With Container Storage Module for Observability, you will gain visibility not only on the capacity of the volumes/file shares you manage with Dell CSM CSI (Container Storage Interface) drivers but also their performance in terms of bandwidth, IOPS, and response time.
Thanks to pre-packaged Grafana dashboards, you will be able to go through these metrics history and see the topology between a Kubernetes PV (Persistent Volume) and its translation as a LUN or file share in the backend array. This module also allows Kubernetes admins to collect array level metrics to check the overall capacity and performance directly from the Prometheus/Grafana tools rather than interfacing directly with the storage system itself.
Metrics data is collected and pushed to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. SSL certificates for TLS between nodes are handled by cert-manager.
Container Storage Module for Observability is composed of several services, each residing in its own GitHub repository, that can be installed following one of the four deployments we support here. Contributions can be made to this repository or any of the Container Storage Module for Observability repositories listed below.
| Name | Repository | Description |
|---|---|---|
| Metrics for PowerFlex | Container Storage Module Metrics for PowerFlex | Metrics for PowerFlex captures telemetry data about Kubernetes storage usage and performance obtained through the CSI (Container Storage Interface) Driver for PowerFlex. The metrics service pushes it to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. Prometheus can then be configured to scrape the OpenTelemetry Collector exporter endpoint to provide metrics, so they can be visualized in Grafana. Please visit the repository for more information. |
| Metrics for PowerStore | Container Storage Module Metrics for PowerStore | Metrics for PowerStore captures telemetry data about Kubernetes storage usage and performance obtained through the CSI (Container Storage Interface) Driver for PowerStore. The metrics service pushes it to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. Prometheus can then be configured to scrape the OpenTelemetry Collector exporter endpoint to provide metrics, so they can be visualized in Grafana. Please visit the repository for more information. |
| Metrics for PowerScale | Container Storage Module Metrics for PowerScale | Metrics for PowerScale captures telemetry data about Kubernetes storage usage and performance obtained through the CSI (Container Storage Interface) Driver for PowerScale. The metrics service pushes it to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. Prometheus can then be configured to scrape the OpenTelemetry Collector exporter endpoint to provide metrics, so they can be visualized in Grafana. Please visit the repository for more information. |
| Metrics for PowerMax | Container Storage Module Metrics for PowerMax | Metrics for PowerMax captures telemetry data about Kubernetes storage usage and performance obtained through the CSI (Container Storage Interface) Driver for PowerMax. The metrics service pushes it to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. Prometheus can then be configured to scrape the OpenTelemetry Collector exporter endpoint to provide metrics, so they can be visualized in Grafana. Please visit the repository for more information. |
| Volume Topology | Container Storage Module Topology | Topology provides Kubernetes administrators with the topology data related to containerized storage that is provisioned by a CSI (Container Storage Interface) Driver for Dell storage products. The Topology service is enabled by default as part of the Container Storage Module for Observability Helm Chart values file. Please visit the repository for more information. |
Container Storage Module for Observability Capabilities
Container Storage Module for Observability provides the following capabilities:
| Capability | PowerMax | PowerFlex | Unity XT | PowerScale | PowerStore |
|---|---|---|---|---|---|
| Collect and expose Volume Metrics via the OpenTelemetry Collector | yes | yes | no | yes | yes |
| Collect and expose File System Metrics via the OpenTelemetry Collector | no | no | no | no | yes |
| Collect and expose export (k8s) node metrics via the OpenTelemetry Collector | no | yes | no | no | no |
| Collect and expose block storage metrics via the OpenTelemetry Collector | yes | yes | no | no | yes |
| Collect and expose file storage metrics via the OpenTelemetry Collector | no | no | no | yes | yes |
| Non-disruptive config changes | yes | yes | no | yes | yes |
| Non-disruptive log level changes | yes | yes | no | yes | yes |
| Grafana Dashboards for displaying metrics and topology data | yes | yes | no | yes | yes |
Topology Data
Container Storage Module for Observability provides Kubernetes administrators with the topology data related to containerized storage. This topology data is visualized using Grafana:
| Field | Description |
|---|---|
| Namespace | The namespace associated with the persistent volume claim |
| Persistent Volume Claim | The name of the persistent volume claim associated with the persistent volume |
| Persistent Volume | The name of the persistent volume |
| Storage Class | The storage class associated with the persistent volume |
| Provisioned Size | The provisioned size of the persistent volume |
| Status | The status of the persistent volume. “Released” indicates the persistent volume does not have a claim. “Bound” indicates the persistent volume has a claim |
| Created | The date the persistent volume was created |
| Storage System | The storage system ID or IP address the volume is associated with |
| Protocol | The storage system protocol type the volume/storage class is associated with |
| Storage Pool | The storage pool name the volume/storage class is associated with |
| Storage System Volume Name | The name of the volume on the storage system that is associated with the persistent volume |
TLS Encryption
Container Storage Module for Observability deployment relies on cert-manager to manage SSL certificates that are used to encrypt communication between various components. When deploying Container Storage Module for Observability, cert-manager is installed and configured automatically. The cert-manager components listed below will be installed alongside Container Storage Module for Observability.
| Component |
|---|
| cert-manager |
| cert-manager-cainjector |
| cert-manager-webhook |
If desired you may provide your own certificate key pair to be used inside the cluster by providing the path to the certificate and key in the Helm chart config. If you do not provide a certificate, one will be generated for you on installation.
NOTE: The certificate provided must be a CA certificate. This is to facilitate automated certificate rotation.
Viewing Logs
Logs can be viewed by using the kubectl logs CLI command to output logs for a specific Pod or Deployment.
For example, the following script will capture logs of all Pods in the CSM namespace and save the output to one file per Pod.
#!/bin/bash
namespace=[CSM_NAMESPACE]
for pod in $(kubectl get pods -n $namespace -o name); do
logFileName=$(echo $pod | tr / -).txt
kubectl logs -n $namespace $pod --all-containers > $logFileName
done