Support testing any application on any k8s/ocp cluster #1133
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nirs
requested review from
raghavendra-talur,
rakeshgm,
ShyamsundarR,
BenamarMk,
ELENAGER,
jmishrarh and
netzzer
nirs
force-pushed
the
test-path
branch
2 times, most recently
from
2db77ca
to
1bc8955
Compare
This comment was marked as outdated.
This comment was marked as outdated.
nirs
commented
Nov 14, 2023
This comment was marked as outdated.
This comment was marked as outdated.
nirs
changed the title
This comment was marked as outdated.
This comment was marked as outdated.
7 tasks
12 tasks
nirs
commented
Nov 27, 2023
Draft
This comment was marked as outdated.
This comment was marked as outdated.
Example run with busybox deployment |
nirs
force-pushed
the
test-path
branch
2 times, most recently
from
189d89a
to
42c093c
Compare
Example run with kubvirt vmStarting the envPreparing the clusterRunning basic testTest config: Deploy: Enable DR: Failover: Relocate: Disable DR: Undeploy: |
Signed-off-by: Nir Soffer <[email protected]>
The script are tiny but it is nice to have verify them with flake8, pylint and black. Signed-off-by: Nir Soffer <[email protected]>
Add "path" config, so test config looks like OpenShift UI:
repo: https://github.com/RamenDR/ocm-ramen-samples.git
path: subscription
branch: main
name: busybox-sample
namespace: busybox-sample
With this we can use basic test to test any subscription based
application in ocm-ramen-samples[1] and ocm-kubevirt-samples[2].
[1] https://github.com/RamenDR/ocm-ramen-samples
[2] https://github.com/aglitke/ocm-kubevirt-samples
Signed-off-by: Nir Soffer <[email protected]>
It is possible now to enable Kubernetes feature gates[1] using minikube
--feature-gates option[2]. We will use this to enable
StatefulSetAutoDeletePVC feature gate.
Example config:
profiles:
- name: featured
feature_gates:
- StatefulSetAutoDeletePVC=true
[1] https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/
[2] https://minikube.sigs.k8s.io/docs/handbook/config/#enabling-feature-gates
Signed-off-by: Nir Soffer <[email protected]>
With this we can create a statesfulset with persistentVolumeClaimRetentionPolicy[1] to have the PVCs deleted when a stateful set is deleted. This policy is required for relocate, otherwise ramen get stuck waiting for vrs to become secondary. [1] https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention Signed-off-by: Nir Soffer <[email protected]>
We use `--namespace ramen-system` but these are deployed in cluster scope. I guess the `--namespace` is ignored in this case since this code works as is. Signed-off-by: Nir Soffer <[email protected]>
Add `pvc_label` configuration so we can test any application. With this we can run basic-test with vms from ocm-kubeivrt-samples[1]. [1] https://github.com/aglitke/ocm-kubevirt-samples Signed-off-by: Nir Soffer <[email protected]>
Previously the dr policy name was hard coded, which works for drenv default dr policy but not for OpenShift clusters. To run with OpenShift cluster you can either change the dr_policy configuration to match it to the actual dr policy, or create a new policy manually before running the tests. Signed-off-by: Nir Soffer <[email protected]>
Since we can test any application now (e.g. busybox, kubvirt), we don't want to mention busybox in the logs. Use config['name'] when we can to make the logs more clear. Signed-off-by: Nir Soffer <[email protected]>
To run tests concurrently, we cannot delete the channel when we deploy an application, so the test cannot be responsible for deploying and deleting the channel. Creating a channel for the tests is now a manual step documented in docs/testing.md and docs/user-quick-start.md. Signed-off-by: Nir Soffer <[email protected]>
basic-test can be run now with custom configuration file. This can be
used to run multiple tests concurrently.
test/basic-test/run --config rbd-deploy.yaml $env 2>rbd.log &
test/basic-test/run --config cephfs-deploy.yaml $env 2>cephfs.log &
wait
Signed-off-by: Nir Soffer <[email protected]>
nirs
force-pushed
the
test-path
branch
2 times, most recently
from
304d635
to
b2c0f54
Compare
nirs
commented
Mar 13, 2024
Add tests configuration for the current subscriptions in
ocm-ramen-samples. We will add more configurations when new
subscriptions are added.
To run test using a custom configuration use:
basic-test/run \
--config configs/deployment-k8s-regional-rbd \
env.yaml
The default config.yaml is a symlink to the new configuration to keep
developers flows working.
Developers can create their own test configuration without modifying the
source:
cp configs/deployment-k8s-regional-rbd.yaml ~/test/myconfig.yaml
# edit myconfig.yaml...
basic-test/run -c ~/test/myconfig.yaml envs/regional-dr.yaml
Signed-off-by: Nir Soffer <[email protected]>
ShyamsundarR
approved these changes
Mar 13, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add support for testing new applications from multiple repos, on multiple cluster types, using different storage.
Status:
Note: To test on odr we need more subscription overlays in ocm-ramen-samples.