Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] support create multiple celeborn clusters(for flink) in one kubernetes namespace #1762

Closed
mddxhj opened this issue Jul 26, 2023 · 1 comment

Comments

@mddxhj
Copy link
Contributor

mddxhj commented Jul 26, 2023

Is your feature request related to a problem? Please describe.

Currently, we can create only one celeborn cluster for flink when deploying on K8S in one namespace. In some scenarios, this is not very convenient. Supposing we have two resource pools divided by business type in the same k8s namespace,we have no way to create a celeborn cluster for both of the two resource pools.

Describe the solution you'd like

For each celeborn cluster, we introduce a single values.yaml file that contains all the necessary configs for creating it, and we can create the cluster using the following helm command

helm install cluster1-celeborn --values cluster1_values.yaml .
helm install cluster2-celeborn --values cluster2_values.yaml .

and we can retrieve k8s pods of each celeborn cluster by using

kubectl get po -l app.kubernetes.io/instance=cluster1-celeborn
kubectl get po -l app.kubernetes.io/instance=cluster2-celeborn

Describe alternatives you've considered

None

@zwangsheng
Copy link
Contributor

zwangsheng commented Aug 22, 2023

Hi @mddxhj, had raised a PR to accomplish this goal.

You can code review whether it meets your expectations, if you have time.

waitinfuture pushed a commit that referenced this issue Aug 22, 2023
…8s namespace

### What changes were proposed in this pull request?
Use built-in helm var {{ .Release.Name }} to naming kubernetes resources, default with user input `celeborn`

close #1762
```
helm install celeborn charts/celeborn
// release name is celeborn
helm install celeborn-beta charts/celeborn
// release name is celeborn-beta
```

### Why are the changes needed?
Support multiple celeborn clusters in the same k8s namespace

### Does this PR introduce _any_ user-facing change?
Yes

### How was this patch tested?
GA and IT

Closes #1804 from zwangsheng/CELEBORN-886.

Authored-by: zwangsheng <[email protected]>
Signed-off-by: zky.zhoukeyong <[email protected]>
(cherry picked from commit 86d78d9)
Signed-off-by: zky.zhoukeyong <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants