Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Worker: Better error handling for credential errors #601

Merged
merged 8 commits into from
Feb 15, 2024

Conversation

josephjclark
Copy link
Collaborator

Closes #600

This is actually a fix down in the engine.

When loading credentials from Lighting, if any credentials fail to load, we catch and report the error, and throw in the main validation function.

Previously, because of how credentials are loaded in nested promises, this case resulted in an uncaught exception and everything would blow up. It's more stable now - and we should get a nice error message to explain the fail reason!

@josephjclark josephjclark changed the base branch from main to release/1-0 February 14, 2024 14:45
@josephjclark
Copy link
Collaborator Author

Hey @taylordowns2000 are you able to retest against this branch?

@josephjclark
Copy link
Collaborator Author

I am unable to reproduce the original issue on my lightning.

The changes here are well worth merging IMO.

As a result of this branch, Taylor sees these logs (edited for clarity):


[RTE] ❯ Loading credential a671b76f-8173-4564-b354-64695aa60240 for step 4415c14f-bb32-4407-a38a-cacf1a7fc49a
[SRV] ✘ error connecting to run:9f9c471a-79f1-4627-bd2f-17b9183ff2c1 {
  "reason": "unauthorized"
}
[RTE] ❯ Error loading credential a671b76f-8173-4564-b354-64695aa60240
[SRV] ✘ ERROR in workflow-error handler: timeout

I think that when the credential fails to load, the worker gets kicked out of the websocket at the lightning end. It is no longer allowed into the run channel.

When the credential fails to load, the worker will try to send logs and events to Lightning. But these fail because the channel is dead - and so we get the timeout on the workflow-error handler (which really means run-error, which has been fixed on this branch)

@taylordowns2000 taylordowns2000 merged commit ab156dc into release/1-0 Feb 15, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Worker: no error handling on fetch:credential
2 participants