Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: health check waits for tasks to finish before becoming unhealthy #788

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

aneojgurhem
Copy link
Contributor

@aneojgurhem aneojgurhem commented Oct 30, 2024

Motivation

Avoid stopping compute plane due to unhealthy health check when tasks are being executed.

Description

Pollster health check has been changed to become unhealthy when there is no tasks in processing in the pollster. Shut down while not be triggered when health check becomes unhealthy, allowing the current tasks to continue. However, no new tasks will be taken from the queue storage. If the tasks does not access the storages during the time they are not available, it will not have any impact on the tasks. If a storage is missing at this point, the tasks should be set to error.

Testing

Unit tests were added. Unit tests and integration tests are passing.

Impact

No impact on infrastructure. If there are connection loss during the execution of the task and the connection comes back before the end of the task, the task should succeed.

Checklist

  • My code adheres to the coding and style guidelines of the project.
  • I have performed a self-review of my code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • I have thoroughly tested my modifications and added tests when necessary.
  • Tests pass locally and in the CI.
  • I have assessed the performance impact of my modifications.

@aneojgurhem aneojgurhem force-pushed the jg/hc branch 2 times, most recently from 7c5a3e5 to 564c0b5 Compare October 30, 2024 15:12
@aneojgurhem aneojgurhem marked this pull request as ready for review October 30, 2024 16:17
lemaitre-aneo
lemaitre-aneo previously approved these changes Oct 30, 2024
Common/tests/Pollster/PollsterTest.cs Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Review in progress
Development

Successfully merging this pull request may close these issues.

2 participants