Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to discard events under load #849

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ananthakumaran
Copy link

@ananthakumaran ananthakumaran commented Jan 9, 2025

Closes #848.

Copy link
Collaborator

@whatyouhide whatyouhide left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fanstastic work.

@@ -319,6 +329,10 @@ defmodule Sentry.LoggerHandler do
config.rate_limiting && RateLimiter.increment(handler_id) == :rate_limited ->
:ok

config.discard_threshold > 0 &&
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit, these are both strictly booleans so you should do

Suggested change
config.discard_threshold > 0 &&
config.discard_threshold > 0 and

Comment on lines +95 to +96
*since v10.8.2* - The number of queued events after which this handler will start
to discard events. If you don't want to discard, set this option to `0`. This option
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be worth documenting how this interacts with :sync_threshold?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discard_threshold takes precedence over sync_threshold:

  • If discard_threshold is set lower than sync_threshold, events will be dropped, and the logger handler will not switch to sync mode.
  • If discard_threshold is set higher than sync_threshold, the logger handler will switch to sync mode, but once the discard_threshold is reached, events will be discarded instead of being queued.

Would this work?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m unclear on the second point—if discard_threshold is higher, we will stop queuing events when we reach sync_threshold. Those events will actually block? I am not sure these options are compatible with each other basically.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, I was assuming queue counter was always increased, but that's not the case. So the counter will never go over the sync_threshold.

discard_threshold and sync_threshold operate independently, with the lower threshold determining the behavior:

  • If discard_threshold is lower than sync_threshold, events will be discarded once the discard_threshold is reached, and the logger handler will not switch to sync mode.
  • If sync_threshold is lower than discard_threshold, the logger handler will switch to sync mode once the sync_threshold is reached, and events will no longer be queued, meaning the discard_threshold will not be reached.

would this work? it seems a bit wordy.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow up question, should I set the default to 0 then? 500 doesn't make sense considering the sync_threshold is 100

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add ability to drop errors instead of going into sync mode
2 participants