cordinator(ticdc): Fix Puller Resolved TS Lag Calculation and Deprecate current_ts Field in Stats

[wlwilliamx]

What problem does this PR solve?

Issue Number: close #11561

What is changed and how it works?

Summary:

This PR addresses a critical issue in the Puller Resolved TS Lag metric, where the lag was incorrectly displayed as -55 years. The root cause of the problem was the improper initialization of the CurrentTs field in the Stats structure within table.pb.go. This led to a situation where CurrentTs remained zero while ResolvedTs was correctly populated, resulting in a negative lag calculation that erroneously reflected a time difference of 55 years (from 1970 to the current year).

Detailed Explanation:

• Issue Background:

  • The Puller Resolved TS Lag was calculated as the difference between CurrentTs and ResolvedTs from the Stats structure.
  • However, the NewReplicationSet function did not correctly initialize CurrentTs, leading to incorrect lag values when metrics were collected.

• Design Intent of current_ts:

  • The current_ts field was initially designed to record the time at the processor side, which would then be used by the coordinator for lag calculation. This design was meant to avoid including the transmission delay and processing time on the coordinator side, as these could introduce inaccuracies in the lag calculation.
  • However, this potential inaccuracy is minimal since the communication between the processor and coordinator happens within the TiCDC cluster, where network delays are typically in the millisecond range. Given that the lag metric is reported in seconds, any millisecond-level discrepancy is considered acceptable.
  • Deprecating the current_ts field may introduce a slight inaccuracy in the lag calculation, as the "current time" will now be fetched directly by the coordinator rather than from the processor. This could result in a minor error due to transmission delays and processing time. However, this inaccuracy is negligible for practical purposes, as the delay is typically within milliseconds and does not significantly impact the metric, which is measured in seconds.
  • Additionally, since this field was part of the protobuf (pb) structure, it added extra overhead to every network transmission by carrying an additional value. It also introduced more complexity into the code, requiring maintenance of this field, which was part of the issue that led to the incorrect metric calculations in this case.

• Root Cause:

  • The use of CurrentTs from the Stats structure was flawed because this timestamp was not reliably up-to-date.
  • The metric calculation requires the "current time," which should always be fetched directly from PD to ensure accuracy, rather than relying on a potentially outdated CurrentTs stored in the Stats structure.

• Solution Implemented:

  • This PR updates the metric calculation to fetch the latest "current time" directly from PD whenever calculating the lag.
  • All references to the CurrentTs field within the codebase, particularly in metric calculations, have been updated to reflect this change.

• Deprecation of current_ts Field:

  • The current_ts field in the Stats structure, defined in cdc/processor/tablepb/table.proto, has been marked as deprecated.
  • The Stats structure is used for data transmission between the Coordinator and Processor, where current_ts was unnecessarily included in the state for each Table Span.
  • This PR removes all dependencies on current_ts within the project, ensuring that the Coordinator and Processor no longer rely on this outdated field.

Impact:

This PR ensures accurate lag calculation in the Puller Resolved TS Lag metric and removes the outdated current_ts field, improving both performance and maintainability by reducing unnecessary transmission overhead and field maintenance. While this change may introduce a slight inaccuracy in the lag calculation due to the use of coordinator-side timestamps, this impact is minimal and well within acceptable thresholds.

Check List

Tests

• Unit test
• Integration test

Questions

Will it cause performance regression or break compatibility?

No. The deprecation of the current_ts field in the Stats structure is backward-compatible because the field is not actively used elsewhere in the project beyond the affected metrics. The existing field remains in the protocol buffer definition but is marked as deprecated, so it won’t break compatibility with older versions that still use the field.

Do you need to update user documentation, design documentation or monitoring documentation?

No.

Release note

Fix the issue that the puller resolved ts lag metric shows incorrect values. #11561

[wlwilliamx]

/cc @3AceShowHand @asddongmen

[codecov]

Codecov Report

Attention: Patch coverage is 27.27273% with 8 lines in your changes missing coverage. Please review.

Project coverage is 55.3542%. Comparing base (25676cf) to head (2519fa6).
Report is 8 commits behind head on master.

Additional details and impacted files

Components	Coverage Δ
cdc	59.9744% <27.2727%> (-1.2712%)	⬇️
dm	50.1039% <ø> (-0.9517%)	⬇️
engine	53.3903% <ø> (-10.0047%)	⬇️

Flag	Coverage Δ
unit	55.3542% <27.2727%> (-2.1367%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master     #11624        +/-   ##
================================================
- Coverage   57.4909%   55.3542%   -2.1367%     
================================================
  Files           851       1001       +150     
  Lines        126486     136349      +9863     
================================================
+ Hits          72718      75475      +2757     
- Misses        48377      55386      +7009     
- Partials       5391       5488        +97     

[wlwilliamx]

/retest

[wlwilliamx]

/retest

[wlwilliamx]

/retest

[wlwilliamx]

/test cdc-integration-storage-test

[wlwilliamx]

/test cdc-integration-mysql-test

[wlwilliamx]

/retest

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2024-09-25 07:25:20.941338528 +0000 UTC m=+1637190.681762466: ☑️ agreed by 3AceShowHand.
• 2024-10-10 08:03:45.437355728 +0000 UTC m=+1119580.857568735: ☑️ agreed by asddongmen.

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-6.5: #11643.

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-7.1: #11644.

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-7.5: #11645.

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-8.1: #11646.