Report total backlog in unbounded source sdf wrapper #34420
Conversation
|
|
||
| public abstract @Nullable Integer getKey(); |
There was a problem hiding this comment.
When I was tinkering with this I had put this down as byte[] or ByteString in case sub-splitting a split is ever added. The root split starts out with an empty byte blob (for a single root) or a varint encoded index (for multiple roots) and the id of a split would be composed of the parent split id and another varint encoded index (constructed when looping over the result of #split()) appended/prepended.
It's entirely unnecessary to model it like that today though and an integer should suffice. :)
stankiewicz
changed the title
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
|
Run Java_Examples_Dataflow_Java21 PreCommit |
|
Run Java_Amazon-Web-Services2_IO_Direct PreCommit |
|
Run Java PreCommit |
|
Run Java_Examples_Dataflow_Java21 PreCommit |
unbounded source sdf wrapper had only reported per split backlog which is limiting reporting for some of the IOs like JMS.
with this change Total Backlog Bytes is being pulled for first split, similarly to v1 implementation.