Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "sleep" feature to the Docker image as a separate binary or as an argument #656

Open
KIVagant opened this issue Mar 8, 2024 · 3 comments
Labels
community Issues or PRs opened by an external contributor help wanted Issues identified as good community contribution opportunities refined Issues that are ready to be prioritized

Comments

@KIVagant
Copy link

KIVagant commented Mar 8, 2024

Is your feature request related to a problem? Please describe.

I'm trying to start using the exporter close to how it is described in this article. I launch the Exporter as a sidecar container to Nginx in a Kubernetes Pod.

But I have a problem. The Nginx in my setup is also a sidecar to the backend container. And I use the preStop container lifecycle hook. It's a simple "exec" command that runs "sleep". This allows to mitigate some 5xx errors for the end-users.

I tried to configure a similar preStop hook for the Exporter. Unfortunately, there's no binary in the Docker image that I can call to run the sleep X command. I wanted to use something like this:

lifecycle:
  preStop:
    exec:
      command:
        - sleep
        - 10

This leads to a problem that Kubernetes may kill the Exporter container before the Nginx container. And a portion of very important metrics will never be exported to the monitoring system. A lot of corner cases appear when a Kubernetes Pod goes down and a new one starts as a replacement. Accurate monitoring is crucial to debug such cases.

The Kubernetes developers introduced a Feature Gate called PodLifecycleSleepAction, which is described here and its goal is basically to replicate that sleep command. The problem is that the Feature Gate current status is alpha and it's available since Kubernetes 1.29. Cloud platforms, such as AWS, don't allow Alpha Gates in their Kubernetes implementations. It can take a year or more for this feature to land into the EKS world.

Describe the solution you'd like

It would be very nice if the "sleep" feature was included into the Docker image of nginx-prometheus-exporter either as a separate command, or as a part of the binary itself, i.e. an argument nginx-prometheus-exporter --sleep 10 that would simply return the 0 exit code after the sleep.

Describe alternatives you've considered

As mentioned above, the PodLifecycleSleepAction is the best alternative. Although, it will take too long for the feature to become available in some production environments with long update lifecycles. In my case we aren't even close to Kubernetes 1.29.

Additional context

This comment was marked as outdated.

@github-actions github-actions bot added the stale Pull requests/issues with no activity label Jun 7, 2024

This comment was marked as outdated.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jun 17, 2024
@lucacome lucacome reopened this Jun 20, 2024
@lucacome lucacome added community Issues or PRs opened by an external contributor and removed stale Pull requests/issues with no activity labels Jun 20, 2024
@mpstefan
Copy link

Hey @KIVagant, we think this makes sense, especially as the feature provided by Kubernetes is not out of beta yet even if you had the latest version. As this project is on minimum maintenance, is there a chance you could take a look at it and add what is needed to get what you want?

Otherwise, I'll stick a help wanted label on this for now.

@mpstefan mpstefan added help wanted Issues identified as good community contribution opportunities backlog Pull requests/issues that are backlog items refined Issues that are ready to be prioritized labels Aug 14, 2024
@mpstefan mpstefan removed the backlog Pull requests/issues that are backlog items label Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community Issues or PRs opened by an external contributor help wanted Issues identified as good community contribution opportunities refined Issues that are ready to be prioritized
Projects
None yet
Development

No branches or pull requests

3 participants