[FLINK-24943][Connectors / Kinesis] Explicitly create KryoSerializer for SequenceNumber class in Kinesis Consumer #20257
Conversation
…eSerializer` that explicitly assigns `KryoSerializer` to `SequenceNumber` class and use this method to initialize state 2. Create unit test that checks for compatibility of previous `TypeInformation` based serializer and explicitly created `KryoSerializer` 3. Replace `Mokito` calls with utility `MockStreamingRuntimeContext` to improve testability and unify initialization method of Streaming Runtime Context
…eSerializer` that explicitly assigns `KryoSerializer` to `SequenceNumber` class and use this method to initialize state 2. Create unit test that checks for compatibility of previous `TypeInformation` based serializer and explicitly created `KryoSerializer` 3. Replace `Mokito` calls with utility `MockStreamingRuntimeContext` to improve testability and unify initialization method of Streaming Runtime Context
|
The code changes look good. Recognise that we specify a KryoSerializer instead of making the SequenceNumber a POJO to maintain backwards compatibility. Given that we are changing the state serializer, could we do a sanity test to check the following?
|
Yes, I can work on it. |
|
@Cyberness please also rebase and squash commits, and update commit message as per the contribution guide |
|
Does it still make sense to work on merging this change considering FLINK-30557? |
|
We have now moved the consumer here https://github.com/apache/flink-connector-aws as part of the connector externalisation effort! |
|
This PR is being marked as stale since it has not had any activity in the last 180 days. If you are having difficulty finding a reviewer, please reach out to the [community](https://flink.apache.org/what-is-flink/community/). If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 90 days, it will be automatically closed. |
What is the purpose of the change
Create
KryoSerializerforSequenceNumberclass explicitly to allow disabling of Generic Types via disableGenericTypes() call.Brief change log
createShardsStateSerializerthat explicitly assignsKryoSerializertoSequenceNumberclass and use this method to initialize stateTypeInformationbased serializer and explicitly createdKryoSerializerMokitocalls with utilityMockStreamingRuntimeContextto improve testability and unify initialization method of Streaming Runtime ContextVerifying this change
Added unit test to verify changed initialization of state.
Does this pull request potentially affect one of the following parts:
Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no
Documentation
Does this pull request introduce a new feature?: no
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented): no