How to conduct CSI tests for RedHat certification
-
Have a healthy OpenShift Cluster somewhere with CLI and UI access to it.
-
Install
OpenShift VirtualizationviaOperatorHub
-
Install
DirectPV Helm OperatorwithOLM:- Use
--insecure-skip-tls-verify=trueif you don't have the certificate of the cluster
- Use
<oc-login> --insecure-skip-tls-verify=true
operator-sdk run bundle quay.io/minio/directpv-operator-bundle:v4.0.10Expected output is:
$ operator-sdk run bundle quay.io/minio/directpv-operator-bundle:v4.0.10
INFO[0014] Creating a File-Based Catalog of the bundle "quay.io/minio/directpv-operator-bundle:v4.0.10"
INFO[0015] Generated a valid File-Based Catalog
INFO[0021] Created registry pod: quay-io-minio-directpv-operator-bundle-v4-0-10
INFO[0022] Created CatalogSource: minio-directpv-operator-catalog
INFO[0022] Created Subscription: minio-directpv-operator-v4-0-10-sub
INFO[0051] Approved InstallPlan install-twgx6 for the Subscription: minio-directpv-operator-v4-0-10-sub
INFO[0051] Waiting for ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" to reach 'Succeeded' phase
INFO[0051] Waiting for ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" to appear
INFO[0054] Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: Pending
INFO[0056] Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: InstallReady
INFO[0057] Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: Installing
INFO[0077] Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: Succeeded
INFO[0077] OLM has successfully installed "minio-directpv-operator.v4.0.10"- Deploy DirectPV using the Operator's CRD.
File:
https://github.com/minio/directpv/blob/master/operator/examples/my-directpv.yaml
kubectl apply -f - <<EOF
apiVersion: charts.quay.io/v1alpha1
kind: DirectPVChart
metadata:
name: directpvchart-sample
spec:
# Default values copied from <project_dir>/helm-charts/directpv-chart/values.yaml
affinity: {}
fullnameOverride: ""
image:
pullPolicy: IfNotPresent
imagePullSecrets: []
nameOverride: ""
nodeSelector: {}
podAnnotations: {}
podSecurityContext: {}
replicaCount: 1
resources: {}
securityContext: {}
serviceAccount:
annotations: {}
create: true
name: ""
tolerations: []
EOF
- Give permissions (wait 5 minutes for service accounts to appear or rollout
DeploymentandDaemonSet):
$ oc get serviceaccount -n directpv
NAME SECRETS AGE
builder 1 115s
default 1 115s
deployer 1 115s
directpv-min-io 1 115sService Accounts with permissions:
oc adm policy add-scc-to-user privileged -n directpv -z builder
oc adm policy add-scc-to-user privileged -n directpv -z default
oc adm policy add-scc-to-user privileged -n directpv -z deployer
oc adm policy add-scc-to-user privileged -n directpv -z directpv-min-io- Discover and init the drives:
kubectl directpv discover
kubectl directpv init drives.yaml --dangerous- Be ready to provide volumes:
$ kubectl directpv info
┌──────┬──────────┬───────────┬─────────┬────────┐
│ NODE │ CAPACITY │ ALLOCATED │ VOLUMES │ DRIVES │
├──────┼──────────┼───────────┼─────────┼────────┤
│ • 25 │ 8.1 TiB │ 0 B │ 0 │ 7 │
│ • 26 │ 9.1 TiB │ 0 B │ 0 │ 7 │
│ • 27 │ 9.6 TiB │ 0 B │ 0 │ 8 │
└──────┴──────────┴───────────┴─────────┴────────┘
0 B/27 TiB used, 0 volumes, 22 drives- In a RHEL Host get the
kubeconfig:
oc config view --raw > kubeconfig.yaml- Depending on the case: https://rhcert.connect.redhat.com/#/cases/03629102 get these two files below accordingly:
https://github.com/cniackz/directpvcertification/blob/main/manifest.yamlhttps://github.com/cniackz/directpvcertification/blob/main/storageclass.yaml
- Execute the tests:
podman login registry.redhat.io
podman run --privileged -v `pwd`:/data --rm -it registry.redhat.io/openshift4/ose-tests sh -c "KUBECONFIG=/data/kubeconfig.yaml TEST_CSI_DRIVER_FILES=/data/manifest.yaml /usr/bin/openshift-tests run openshift/csi --junit-dir /data/results"Expected output is:
Suite run returned error: 1 fail, 40 pass, 213 skip (6m27s)
Storage Capabilities (guaranteed only on full CSI test suite with 0 fails)
==========================================================================
Driver short name: directpv
Driver name: [directpv.min.io](http://directpv.min.io/)
Storage class: storageclass.yaml
Supported OpenShift / CSI features:
Persistent volumes: true
Raw block mode: false
FSGroup: true
Executable files on a volume: true
Volume snapshots: false
Volume cloning: false
Use volume from multiple pods on a node: false
ReadWriteMany access mode: false
Volume expansion for controller: false
Volume expansion for node: true
Volume limits: false
Volume can run on single node: true
Topology: true
Supported OpenShift Virtualization features:
Raw block VM disks: false
Live migration: false
VM snapshots: false
Storage-assisted cloning: false
error running options: 1 fail, 40 pass, 213 skip (6m27s)error: 1 fail, 40 pass, 213 skip (6m27s)- Then compress the results:
tar czf results_8.tar.gz results- Save the results and upload them to the case for John to see.
Celis normally saves them on this repo, example:
https://github.com/cniackz/directpvcertification/blob/main/results-6.tar.gz
- Couple of times, we have seen that after executing CSI Tests, cluster becomes unstable and some nodes are unresponsive due to some kubevirt heartbeat; could be related but more root cause is needed.
