Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add pipeline status monitor #372

Merged
merged 2 commits into from
Dec 7, 2023
Merged

Conversation

nicHoch
Copy link
Collaborator

@nicHoch nicHoch commented Nov 14, 2023

Status logger and notification script for the pipeline.

SetUp via cron.
Query the number of open files still to process. Logs that number into a status file
and checks if the the number is constantly equal or increasing.

Sends an notification via mail if a possible pipeline stuck is detected.

@nicHoch
Copy link
Collaborator Author

nicHoch commented Nov 20, 2023

@samaloney "unfortunately" we missed a stuck case on this weekend. i waited until i get the mail but forgot that i changed to another branch in me dev environment ...
but if you review it i can merge it into master soon

Copy link
Collaborator

@samaloney samaloney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is really a script maybe belongs in script folder rather than here?

@nicHoch
Copy link
Collaborator Author

nicHoch commented Dec 7, 2023

all our entry-points / mains clis for regular use are in this processing folder. the script are more used as helpers on times ...

i will also add a entry point in the project setup

@codecov-commenter
Copy link

codecov-commenter commented Dec 7, 2023

Codecov Report

Attention: 61 lines in your changes are missing coverage. Please review.

Comparison is base (a801e54) 77.79% compared to head (3b00fa6) 77.12%.

Files Patch % Lines
stixcore/processing/pipeline_monitor.py 0.00% 59 Missing ⚠️
stixcore/processing/pipeline_status.py 33.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #372      +/-   ##
==========================================
- Coverage   77.79%   77.12%   -0.68%     
==========================================
  Files          65       66       +1     
  Lines        6777     6836      +59     
==========================================
  Hits         5272     5272              
- Misses       1505     1564      +59     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@nicHoch nicHoch merged commit f065afe into i4Ds:master Dec 7, 2023
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants