Skip to content

Generate correctly the MinIO scrape configs for Multi Tenants scenarios #2456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

JoelRuizRojas
Copy link

@JoelRuizRojas JoelRuizRojas commented May 27, 2025

Description

Next fixes introduced:

  1. Prometheus scrape config job names were composed by the tenant name only, which makes it difficult to handle multi tenants with same name different namespace. Namespace was added for the MinIO scrape config job names in the secret minio-prom-additional-scrape-config to fix this issue.
  2. In Multi tenant scenarios where one is prometheus enabled and another is not, issues updating and deleting the scrape configs was constantly happening in every default tenant reconciliation loop (5 min).
  3. Added the same permissions for prometheusagents resources (as with Helm) when deploying the Operator using kustomize.
  4. When a tenant is deleted, allow it to be enqueued for controller reconciliation so that the tenant Prometheus scrape configs can be deleted.

A few unit tests were added to cover points (1) and (2) above.

Note: I need to port this fix to AIStor

Related Issue

Fixing issues:

#2425
#2218

Type of Change

  • Bug fix 🐛
  • New feature 🚀
  • Breaking change 🚨
  • Documentation update 📖
  • Refactor 🔨
  • Other (please describe) ⬇️

Screenshots (if applicable e.g before/after)

Checklist

  • I have tested these changes
  • I have updated relevant documentation (if applicable)
  • I have added necessary unit tests (if applicable)

Test Steps

  1. Build operator image, and modify Helm deployment operator values to use the locally built operator image
  2. kind load docker-image minio/operator:noop (loads locally built image to kind cluster)
  3. helm install minio-operator helm/operator --namespace minio-operator --create-namespace (install Operator)
  4. helm install prometheus-operator prometheus-community/kube-prometheus-stack(installs prometheus controllers stack and grafana controller)
  5. helm install my-tenant helm/tenant --namespace tenant-ns --create-namespace (installs a tenant in a given namespace)
  6. helm install my-tenant helm/tenant --namespace tenant-ns --create-namespace (installs a tenant in another given namespace)
  7. Edit the tenants so that prometheusOperator: true
  8. Wait for operator to create and update the Prometheus scrape configs at secret minio-prom-additional-scrape-config
  9. Play around deleting tenants or setting prometheusOperator: false so that the tenant scrape configs are removed from minio-prom-additional-scrape-config secret. Or add more metric endpoint with tenant config PrometheusOperatorScrapeMetricsPaths and verify the scrape configs are added to the secret.

Additional Notes / Context

@JoelRuizRojas JoelRuizRojas changed the title Generate correctly the MinIO scrape configs for Multi Tenants escenarios Generate correctly the MinIO scrape configs for Multi Tenants scenarios May 27, 2025
Copy link
Contributor

@cniackz cniackz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants