-
Notifications
You must be signed in to change notification settings - Fork 720
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
client: make TSO client request duration include failed requests #8410
Conversation
Signed-off-by: JmPotato <[email protected]>
/test build |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #8410 +/- ##
==========================================
- Coverage 77.21% 77.20% -0.01%
==========================================
Files 471 471
Lines 61835 61840 +5
==========================================
- Hits 47747 47745 -2
- Misses 10499 10501 +2
- Partials 3589 3594 +5
Flags with carried forward coverage won't be shown. Click here to find out more. |
client/tso_stream.go
Outdated
@@ -141,6 +141,7 @@ func (s *pdTSOStream) processRequests( | |||
} | |||
tsoBatchSendLatency.Observe(time.Since(batchStartTime).Seconds()) | |||
resp, err := s.stream.Recv() | |||
requestDurationTSO.Observe(time.Since(start).Seconds()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to distinguish them?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mean to distinguish the duration between success and failure? Currently, I don't think it's necessary because we generally only use this metric to compare TSO Wait Duration
to infer the source of the delay, the latter does not distinguish between successful and failed requests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I was wondering if we find a latency spike, we might need to know if the request has succeeded or failed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make sense. Addressed. PTAL.
Signed-off-by: JmPotato <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please paste a pic to show the metric :)
cmdFailedDurationGet = cmdFailedDuration.WithLabelValues("get") | ||
cmdFailedDurationPut = cmdFailedDuration.WithLabelValues("put") | ||
cmdFailedDurationUpdateGCSafePointV2 = cmdFailedDuration.WithLabelValues("update_gc_safe_point_v2") | ||
cmdFailedDurationUpdateServiceSafePointV2 = cmdFailedDuration.WithLabelValues("update_service_safe_point_v2") | ||
|
||
requestDurationTSO = requestDuration.WithLabelValues("tso") | ||
requestFailedDurationTSO = requestDuration.WithLabelValues("tso-failed") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need underline?
requestFailedDurationTSO = requestDuration.WithLabelValues("tso-failed") | |
requestFailedDurationTSO = requestDuration.WithLabelValues("tso_failed") |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: HuSharp, rleungx The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/hold |
/unhold |
What problem does this PR solve?
Issue Number: ref #8281.
What is changed and how does it work?
Check List
Tests
Release note