Log stack trace when refresh cache sync recovers from panic #5623
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Jason Parraga <[email protected]>
Signed-off-by: Jason Parraga <[email protected]>
Sovietaced
force-pushed
the
refresh-cache-panic
branch
from
960030a to
878b86e
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #5623 +/- ##
=======================================
Coverage 36.16% 36.17%
=======================================
Files 1302 1302
Lines 109405 109405
=======================================
+ Hits 39568 39578 +10
+ Misses 65701 65690 -11
- Partials 4136 4137 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Signed-off-by: Jason Parraga <[email protected]>
Sovietaced
force-pushed
the
refresh-cache-panic
branch
from
813d45f to
169baa9
Compare
bgedik
pushed a commit
to bgedik/flyte
that referenced
this pull request
Aug 15, 2024
…#5623) Signed-off-by: Bugra Gedik <[email protected]>
vlibov
pushed a commit
to vlibov/flyte
that referenced
this pull request
Aug 16, 2024
…#5623) Signed-off-by: Vladyslav Libov <[email protected]>
eapolinario
added a commit
that referenced
this pull request
Aug 20, 2024
…ame (#5616) * Add environment variable for pod name Signed-off-by: Bugra Gedik <[email protected]> * [flyteadmin] Refactor panic recovery into middleware (#5546) * Refactor panic handling to middleware Signed-off-by: Jason Parraga <[email protected]> * Remove registration of old panicCounter Signed-off-by: Jason Parraga <[email protected]> * Add test coverage Signed-off-by: Jason Parraga <[email protected]> --------- Signed-off-by: Jason Parraga <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Snowflake agent Doc (#5620) * TEST build Signed-off-by: Future-Outlier <[email protected]> * remove emphasize-lines Signed-off-by: Future-Outlier <[email protected]> * test build Signed-off-by: Future-Outlier <[email protected]> * revert Signed-off-by: Future-Outlier <[email protected]> --------- Signed-off-by: Future-Outlier <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * [flytepropeller][compiler] Error Handling when Type is not found (#5612) * FlytePropeller Compiler Avoid Crash when Type not found Signed-off-by: Future-Outlier <[email protected]> * Update pingsu's error message advices Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: pingsutw <[email protected]> * fix lint Signed-off-by: Future-Outlier <[email protected]> * Trigger CI Signed-off-by: Future-Outlier <[email protected]> * Trigger CI Signed-off-by: Future-Outlier <[email protected]> --------- Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: pingsutw <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Fix nil pointer when task plugin load returns error (#5622) Signed-off-by: Bugra Gedik <[email protected]> * Log stack trace when refresh cache sync recovers from panic (#5623) Signed-off-by: Bugra Gedik <[email protected]> * use private-key (#5626) Signed-off-by: Bugra Gedik <[email protected]> * Explain how Agent Secret Works (#5625) * first version Signed-off-by: Future-Outlier <[email protected]> * update Signed-off-by: Future-Outlier <[email protected]> --------- Signed-off-by: Future-Outlier <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Fix typo in execution manager (#5619) Signed-off-by: ddl-rliu <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Amend Admin to use grpc message size (#5628) * add send arg Signed-off-by: Yee Hing Tong <[email protected]> * Add acction to remove cache in gh runner Signed-off-by: Eduardo Apolinario <[email protected]> * Use correct checked out path Signed-off-by: Eduardo Apolinario <[email protected]> * Path in strings Signed-off-by: Eduardo Apolinario <[email protected]> * Checkout repo in root Signed-off-by: Eduardo Apolinario <[email protected]> * Use the correct path to new action Signed-off-by: Eduardo Apolinario <[email protected]> * Do not use gh var in path to clear-action-cache Signed-off-by: Eduardo Apolinario <[email protected]> * Remove wrong invocation of clear-action-cache Signed-off-by: Eduardo Apolinario <[email protected]> * GITHUB_WORKSPACE is implicit in the checkout action Signed-off-by: Eduardo Apolinario <[email protected]> * Refer to local `flyte` directory Signed-off-by: Eduardo Apolinario <[email protected]> --------- Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * document the process of setting ttl for a ray cluster (#5636) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Add CustomHeaderMatcher to pass additional headers (#5563) Signed-off-by: Andrew Dye <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Turn flyteidl and flytectl releases into manual gh workflows (#5635) * Make flyteidl releases go through a manual gh workflow Signed-off-by: Eduardo Apolinario <[email protected]> * Make flytectl releases go through a manual gh workflow Signed-off-by: Eduardo Apolinario <[email protected]> * Rewrite the documentation for `version` and clarify wording in RELEASE.md Signed-off-by: Eduardo Apolinario <[email protected]> --------- Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * docs: fix typo (#5643) * fix CHANGELOG-v0.2.0.md Signed-off-by: Christina <[email protected]> * fix CHANGELOG-v1.0.2-b1.md Signed-off-by: Christina <[email protected]> * fix CHANGELOG-v1.1.0.md Signed-off-by: Christina <[email protected]> * fix CHANGELOG-v1.3.0.md Signed-off-by: Christina <[email protected]> --------- Signed-off-by: Christina <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Use enable_deck=True in docs (#5645) Signed-off-by: Bugra Gedik <[email protected]> * Fix flyteidl release checkout all tags (#5646) * Fetch all tags in flyteidl-release.yml Signed-off-by: Eduardo Apolinario <[email protected]> * Fix sed expression for npm job Signed-off-by: Eduardo Apolinario <[email protected]> --------- Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Install pyarrow in sandbox functional tests (#5647) Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * docs: add documentation for configuring notifications in GCP (#5545) * update Signed-off-by: Desi Hsu <[email protected]> * dco Signed-off-by: Desi Hsu <[email protected]> * dco Signed-off-by: Desi Hsu <[email protected]> * typo Signed-off-by: Desi Hsu <[email protected]> --------- Signed-off-by: Desi Hsu <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Correct "sucessfile" to "successfile" (#5652) Signed-off-by: Bugra Gedik <[email protected]> * Fix ordering for custom template values in cluster resource controller (#5648) Signed-off-by: Katrina Rogan <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Don't error when attempting to trigger schedules for inactive projects (#5649) * Don't error when attempting to trigger schedules for inactive projects Signed-off-by: Katrina Rogan <[email protected]> * regen Signed-off-by: Katrina Rogan <[email protected]> --------- Signed-off-by: Katrina Rogan <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * fix tests Signed-off-by: Bugra Gedik <[email protected]> * change to shorter names Signed-off-by: Bugra Gedik <[email protected]> * change to shorter names Signed-off-by: Bugra Gedik <[email protected]> * change to shorter names Signed-off-by: Bugra Gedik <[email protected]> * change to shorter names Signed-off-by: Bugra Gedik <[email protected]> * change to shorter names Signed-off-by: Bugra Gedik <[email protected]> * Fix comment symbol Signed-off-by: Eduardo Apolinario <[email protected]> * fix one more test Signed-off-by: Bugra Gedik <[email protected]> --------- Signed-off-by: Bugra Gedik <[email protected]> Signed-off-by: Jason Parraga <[email protected]> Signed-off-by: Future-Outlier <[email protected]> Signed-off-by: ddl-rliu <[email protected]> Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Eduardo Apolinario <[email protected]> Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Andrew Dye <[email protected]> Signed-off-by: Christina <[email protected]> Signed-off-by: Desi Hsu <[email protected]> Signed-off-by: Katrina Rogan <[email protected]> Co-authored-by: Jason Parraga <[email protected]> Co-authored-by: Future-Outlier <[email protected]> Co-authored-by: pingsutw <[email protected]> Co-authored-by: ddl-rliu <[email protected]> Co-authored-by: Yee Hing Tong <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Co-authored-by: Andrew Dye <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Co-authored-by: Christina <[email protected]> Co-authored-by: Thomas J. Fan <[email protected]> Co-authored-by: desihsu <[email protected]> Co-authored-by: ShengYu <[email protected]> Co-authored-by: Katrina Rogan <[email protected]>
pmahindrakar-oss
pushed a commit
that referenced
this pull request
Sep 9, 2024
…ame (#5616) * Add environment variable for pod name Signed-off-by: Bugra Gedik <[email protected]> * [flyteadmin] Refactor panic recovery into middleware (#5546) * Refactor panic handling to middleware Signed-off-by: Jason Parraga <[email protected]> * Remove registration of old panicCounter Signed-off-by: Jason Parraga <[email protected]> * Add test coverage Signed-off-by: Jason Parraga <[email protected]> --------- Signed-off-by: Jason Parraga <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Snowflake agent Doc (#5620) * TEST build Signed-off-by: Future-Outlier <[email protected]> * remove emphasize-lines Signed-off-by: Future-Outlier <[email protected]> * test build Signed-off-by: Future-Outlier <[email protected]> * revert Signed-off-by: Future-Outlier <[email protected]> --------- Signed-off-by: Future-Outlier <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * [flytepropeller][compiler] Error Handling when Type is not found (#5612) * FlytePropeller Compiler Avoid Crash when Type not found Signed-off-by: Future-Outlier <[email protected]> * Update pingsu's error message advices Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: pingsutw <[email protected]> * fix lint Signed-off-by: Future-Outlier <[email protected]> * Trigger CI Signed-off-by: Future-Outlier <[email protected]> * Trigger CI Signed-off-by: Future-Outlier <[email protected]> --------- Signed-off-by: Future-Outlier <[email protected]> Co-authored-by: pingsutw <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Fix nil pointer when task plugin load returns error (#5622) Signed-off-by: Bugra Gedik <[email protected]> * Log stack trace when refresh cache sync recovers from panic (#5623) Signed-off-by: Bugra Gedik <[email protected]> * use private-key (#5626) Signed-off-by: Bugra Gedik <[email protected]> * Explain how Agent Secret Works (#5625) * first version Signed-off-by: Future-Outlier <[email protected]> * update Signed-off-by: Future-Outlier <[email protected]> --------- Signed-off-by: Future-Outlier <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Fix typo in execution manager (#5619) Signed-off-by: ddl-rliu <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Amend Admin to use grpc message size (#5628) * add send arg Signed-off-by: Yee Hing Tong <[email protected]> * Add acction to remove cache in gh runner Signed-off-by: Eduardo Apolinario <[email protected]> * Use correct checked out path Signed-off-by: Eduardo Apolinario <[email protected]> * Path in strings Signed-off-by: Eduardo Apolinario <[email protected]> * Checkout repo in root Signed-off-by: Eduardo Apolinario <[email protected]> * Use the correct path to new action Signed-off-by: Eduardo Apolinario <[email protected]> * Do not use gh var in path to clear-action-cache Signed-off-by: Eduardo Apolinario <[email protected]> * Remove wrong invocation of clear-action-cache Signed-off-by: Eduardo Apolinario <[email protected]> * GITHUB_WORKSPACE is implicit in the checkout action Signed-off-by: Eduardo Apolinario <[email protected]> * Refer to local `flyte` directory Signed-off-by: Eduardo Apolinario <[email protected]> --------- Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * document the process of setting ttl for a ray cluster (#5636) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Add CustomHeaderMatcher to pass additional headers (#5563) Signed-off-by: Andrew Dye <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Turn flyteidl and flytectl releases into manual gh workflows (#5635) * Make flyteidl releases go through a manual gh workflow Signed-off-by: Eduardo Apolinario <[email protected]> * Make flytectl releases go through a manual gh workflow Signed-off-by: Eduardo Apolinario <[email protected]> * Rewrite the documentation for `version` and clarify wording in RELEASE.md Signed-off-by: Eduardo Apolinario <[email protected]> --------- Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * docs: fix typo (#5643) * fix CHANGELOG-v0.2.0.md Signed-off-by: Christina <[email protected]> * fix CHANGELOG-v1.0.2-b1.md Signed-off-by: Christina <[email protected]> * fix CHANGELOG-v1.1.0.md Signed-off-by: Christina <[email protected]> * fix CHANGELOG-v1.3.0.md Signed-off-by: Christina <[email protected]> --------- Signed-off-by: Christina <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Use enable_deck=True in docs (#5645) Signed-off-by: Bugra Gedik <[email protected]> * Fix flyteidl release checkout all tags (#5646) * Fetch all tags in flyteidl-release.yml Signed-off-by: Eduardo Apolinario <[email protected]> * Fix sed expression for npm job Signed-off-by: Eduardo Apolinario <[email protected]> --------- Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Install pyarrow in sandbox functional tests (#5647) Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * docs: add documentation for configuring notifications in GCP (#5545) * update Signed-off-by: Desi Hsu <[email protected]> * dco Signed-off-by: Desi Hsu <[email protected]> * dco Signed-off-by: Desi Hsu <[email protected]> * typo Signed-off-by: Desi Hsu <[email protected]> --------- Signed-off-by: Desi Hsu <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Correct "sucessfile" to "successfile" (#5652) Signed-off-by: Bugra Gedik <[email protected]> * Fix ordering for custom template values in cluster resource controller (#5648) Signed-off-by: Katrina Rogan <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * Don't error when attempting to trigger schedules for inactive projects (#5649) * Don't error when attempting to trigger schedules for inactive projects Signed-off-by: Katrina Rogan <[email protected]> * regen Signed-off-by: Katrina Rogan <[email protected]> --------- Signed-off-by: Katrina Rogan <[email protected]> Signed-off-by: Bugra Gedik <[email protected]> * fix tests Signed-off-by: Bugra Gedik <[email protected]> * change to shorter names Signed-off-by: Bugra Gedik <[email protected]> * change to shorter names Signed-off-by: Bugra Gedik <[email protected]> * change to shorter names Signed-off-by: Bugra Gedik <[email protected]> * change to shorter names Signed-off-by: Bugra Gedik <[email protected]> * change to shorter names Signed-off-by: Bugra Gedik <[email protected]> * Fix comment symbol Signed-off-by: Eduardo Apolinario <[email protected]> * fix one more test Signed-off-by: Bugra Gedik <[email protected]> --------- Signed-off-by: Bugra Gedik <[email protected]> Signed-off-by: Jason Parraga <[email protected]> Signed-off-by: Future-Outlier <[email protected]> Signed-off-by: ddl-rliu <[email protected]> Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Eduardo Apolinario <[email protected]> Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Andrew Dye <[email protected]> Signed-off-by: Christina <[email protected]> Signed-off-by: Desi Hsu <[email protected]> Signed-off-by: Katrina Rogan <[email protected]> Co-authored-by: Jason Parraga <[email protected]> Co-authored-by: Future-Outlier <[email protected]> Co-authored-by: pingsutw <[email protected]> Co-authored-by: ddl-rliu <[email protected]> Co-authored-by: Yee Hing Tong <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Co-authored-by: Andrew Dye <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]> Co-authored-by: Christina <[email protected]> Co-authored-by: Thomas J. Fan <[email protected]> Co-authored-by: desihsu <[email protected]> Co-authored-by: ShengYu <[email protected]> Co-authored-by: Katrina Rogan <[email protected]> Signed-off-by: pmahindrakar-oss <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request improves the auto refresh cache's
syncpanic recovery logic such that it logs the stack trace for the panic that occurred. This makes it much easier to understand what exactly failed in the cache sync logic (ie. instead of just something like "invalid memory address or nil pointer dereference")