[INLONG-11129][Sort] Enhanced source metric instrumentation for InLong Sort Flink Connector #11130
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
woofyzhao
previously approved these changes
Sep 16, 2024
|
plz provide some test result shortcut in the comment~， thx~ |
Thanks for your question :) The first image shows the newly-added metrics in Flink Web Dashboard. Click To View Image
The second image shows all the metrics reported by Slf4jReporter and the newly-added metrics have been highlighted with red rectangles. Click To View Image
The following log is the metrics copied from the log of the docker container of Flink taskmanager, just for your reference. Click To View Complete Log |
|
Briefly explaining the fixes made to the instantiation of SourceExactlyMetric: When only the MetricOptions parameter is passed to the SourceExactlyMetric constructor, self-defined metrics are not registered with the metricGroup. Therefore, to ensure proper registration of self-defined metrics, it is necessary to retrieve the metricGroup from the runtimeContext within the sourceFunction and pass it as the second parameter to the SourceExactlyMetric constructor. |
|
bugs fixed |
|
@PeterZh6 please fix the failed workflow, thanks. |
Thanks for following up with this. However, I am unable to reproduce the test failure locally, so may still need some time. |
The failure is linked to [Bug][Sort] Mongodb2StarRocksTest Failure Due to Potential Dependency Conflicts #11166 |
PeterZh6
force-pushed
the
INLONG-11129
branch
from
eafc3a2
to
e13b725
Compare
vernedeng
previously approved these changes
Sep 23, 2024
XiaoYou201
previously approved these changes
Sep 23, 2024
...tgres-cdc/src/main/java/org/apache/inlong/sort/postgre/RowDataDebeziumDeserializeSchema.java
Outdated
Show resolved
Hide resolved
...ectors/postgres-cdc/src/main/java/org/apache/inlong/sort/postgre/DebeziumSourceFunction.java
Outdated
Show resolved
Hide resolved
...ectors/postgres-cdc/src/main/java/org/apache/inlong/sort/postgre/DebeziumSourceFunction.java
Outdated
Show resolved
Hide resolved
luchunliang
previously approved these changes
Sep 23, 2024
PeterZh6
dismissed stale reviews from luchunliang, XiaoYou201, and vernedeng
via
ab4420f
PeterZh6
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #11129
Motivation
The primary goal of this PR is to enhance metric instrumentation for the InLong Sort Flink Connector, specifically for the Postgres-CDC connector. This change is aimed at improving observability by introducing additional metrics that track serialization/deserialization, snapshot states, and checkpoint completion.
Modifications
This feature focuses on SourceMetric only
Deserialization Metrics:
Added counters to track successful and failed deserialization attempts (
numDeserializeSuccess,numDeserializeError).Added latency gauge to measure time taken for deserialization (
deserializeTimeLag).SnapshotState Metrics:
Added counters for the number of snapshots created (
numSnapshotCreate) and errors encountered during snapshot operations (numSnapshotError).NotifyComplete Metrics:
Added a counter to track completed snapshots (
numCompletedSnapshots).Added latency gauge for the time between snapshot creation and checkpoint completion (
snapshotToCheckpointTimeLag).Verifying this change
(Please pick either of the following options)
This change is a trivial rework/code cleanup without any test coverage.
This change is already covered by existing tests, such as:
(please describe tests)
This change added tests and can be verified as follows:
Make Flink
takemanagerreport the metrics, with Slf4jReporter for example.Add the following configurations to
conf/flink-conf.yamlof flinktaskmanagerand the above mentioned metrics will be printed to the logging file onceinlong-sortstarts to process data.Documentation