Skip to content

[FLINK-38609][table] Make generation of xml plans deterministic #27183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

[FLINK-38609][table] Make generation of xml plans deterministic #27183

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f0ee7cb
[hotfix][tests] Make editorconfig for plan tests expecting spaces for…
snuyanzin Nov 1, 2025
e8b03e0
[hotfix][table] Remove trailing spaces and fix typo
snuyanzin Nov 3, 2025
71c24da
[FLINK-38609][table] Enable `SubqueryCorrelateVariablesValidationTest…
snuyanzin Nov 3, 2025
5f13169
[FLINK-38609][table] Make generation of xml plans deterministic
snuyanzin Nov 3, 2025
2a36a2f
[FLINK-38609][table] Regenerate plans in deterministic manner
snuyanzin Nov 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .editorconfig
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,11 @@ insert_final_newline = false
indent_style = tab
indent_size = 4

# For verify plan tests
[*Test.xml]
indent_style = space
indent_size = 2
Comment on lines +274 to +276
Copy link
Contributor Author

@snuyanzin snuyanzin Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sync with DiffRepository where indentation is in spaces

Copy link
Contributor

@gustavodemorais gustavodemorais Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great 👍 Also had this indentation issue when manually updating xmls


[*.py]
indent_style = space
indent_size = 4
277 changes: 206 additions & 71 deletions ...link-table-planner/src/test/java/org/apache/flink/table/planner/utils/DiffRepository.java
View file
Open in desktop

Large diffs are not rendered by default.

52 changes: 26 additions & 26 deletions ...e/flink-table-planner/src/test/resources/org/apache/flink/table/api/batch/ExplainTest.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ MultipleInput(readOrder=[0,1], members=[\nHashJoin(joinType=[InnerJoin], where=[
]]>
</Resource>
</TestCase>
<TestCase name="testExplainWithAgg[extended=true]">
<TestCase name="testExplainWithAgg[extended=false]">
Copy link
Contributor Author

@snuyanzin snuyanzin Nov 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and all other xml files: were dropped and completely regenerated

<Resource name="explain">
<![CDATA[== Abstract Syntax Tree ==
LogicalProject(EXPR$0=[$1])
Expand All @@ -103,12 +103,12 @@ LogicalProject(EXPR$0=[$1])
+- LogicalTableScan(table=[[default_catalog, default_database, MyTable1]])

== Optimized Physical Plan ==
Calc(select=[EXPR$0]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- HashAggregate(isMerge=[true], groupBy=[a], select=[a, Final_COUNT(count1$0) AS EXPR$0]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- Exchange(distribution=[hash[a]]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- LocalHashAggregate(groupBy=[a], select=[a, Partial_COUNT(*) AS count1$0]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- Calc(select=[a]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- BoundedStreamScan(table=[[default_catalog, default_database, MyTable1]], fields=[a, b, c]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
Calc(select=[EXPR$0])
+- HashAggregate(isMerge=[true], groupBy=[a], select=[a, Final_COUNT(count1$0) AS EXPR$0])
+- Exchange(distribution=[hash[a]])
+- LocalHashAggregate(groupBy=[a], select=[a, Partial_COUNT(*) AS count1$0])
+- Calc(select=[a])
+- BoundedStreamScan(table=[[default_catalog, default_database, MyTable1]], fields=[a, b, c])

== Optimized Execution Plan ==
Calc(select=[EXPR$0])
Expand All @@ -120,7 +120,7 @@ Calc(select=[EXPR$0])
]]>
</Resource>
</TestCase>
<TestCase name="testExplainWithAgg[extended=false]">
<TestCase name="testExplainWithAgg[extended=true]">
<Resource name="explain">
<![CDATA[== Abstract Syntax Tree ==
LogicalProject(EXPR$0=[$1])
Expand All @@ -129,12 +129,12 @@ LogicalProject(EXPR$0=[$1])
+- LogicalTableScan(table=[[default_catalog, default_database, MyTable1]])

== Optimized Physical Plan ==
Calc(select=[EXPR$0])
+- HashAggregate(isMerge=[true], groupBy=[a], select=[a, Final_COUNT(count1$0) AS EXPR$0])
+- Exchange(distribution=[hash[a]])
+- LocalHashAggregate(groupBy=[a], select=[a, Partial_COUNT(*) AS count1$0])
+- Calc(select=[a])
+- BoundedStreamScan(table=[[default_catalog, default_database, MyTable1]], fields=[a, b, c])
Calc(select=[EXPR$0]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- HashAggregate(isMerge=[true], groupBy=[a], select=[a, Final_COUNT(count1$0) AS EXPR$0]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- Exchange(distribution=[hash[a]]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- LocalHashAggregate(groupBy=[a], select=[a, Partial_COUNT(*) AS count1$0]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- Calc(select=[a]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- BoundedStreamScan(table=[[default_catalog, default_database, MyTable1]], fields=[a, b, c]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}

== Optimized Execution Plan ==
Calc(select=[EXPR$0])
Expand Down Expand Up @@ -446,35 +446,35 @@ SortLimit(orderBy=[a ASC], offset=[0], fetch=[5], global=[true])
]]>
</Resource>
</TestCase>
<TestCase name="testExplainWithTableSourceScan[extended=true]">
<TestCase name="testExplainWithTableSourceScan[extended=false]">
<Resource name="explain">
<![CDATA[== Abstract Syntax Tree ==
LogicalProject(a=[$0], b=[$1], c=[$2])
+- LogicalTableScan(table=[[default_catalog, default_database, MyTable]])

== Optimized Physical Plan ==
TableSourceScan(table=[[default_catalog, default_database, MyTable]], fields=[a, b, c]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
TableSourceScan(table=[[default_catalog, default_database, MyTable]], fields=[a, b, c])

== Optimized Execution Plan ==
TableSourceScan(table=[[default_catalog, default_database, MyTable]], fields=[a, b, c])
]]>
</Resource>
</TestCase>
<TestCase name="testExplainWithTableSourceScan[extended=false]">
<TestCase name="testExplainWithTableSourceScan[extended=true]">
<Resource name="explain">
<![CDATA[== Abstract Syntax Tree ==
LogicalProject(a=[$0], b=[$1], c=[$2])
+- LogicalTableScan(table=[[default_catalog, default_database, MyTable]])

== Optimized Physical Plan ==
TableSourceScan(table=[[default_catalog, default_database, MyTable]], fields=[a, b, c])
TableSourceScan(table=[[default_catalog, default_database, MyTable]], fields=[a, b, c]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}

== Optimized Execution Plan ==
TableSourceScan(table=[[default_catalog, default_database, MyTable]], fields=[a, b, c])
]]>
</Resource>
</TestCase>
<TestCase name="testExplainWithUnion[extended=true]">
<TestCase name="testExplainWithUnion[extended=false]">
<Resource name="explain">
<![CDATA[== Abstract Syntax Tree ==
LogicalUnion(all=[true])
Expand All @@ -484,9 +484,9 @@ LogicalUnion(all=[true])
+- LogicalTableScan(table=[[default_catalog, default_database, MyTable2]])

== Optimized Physical Plan ==
Union(all=[true], union=[a, b, c]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
:- BoundedStreamScan(table=[[default_catalog, default_database, MyTable1]], fields=[a, b, c]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- BoundedStreamScan(table=[[default_catalog, default_database, MyTable2]], fields=[d, e, f]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
Union(all=[true], union=[a, b, c])
:- BoundedStreamScan(table=[[default_catalog, default_database, MyTable1]], fields=[a, b, c])
+- BoundedStreamScan(table=[[default_catalog, default_database, MyTable2]], fields=[d, e, f])

== Optimized Execution Plan ==
Union(all=[true], union=[a, b, c])
Expand All @@ -495,7 +495,7 @@ Union(all=[true], union=[a, b, c])
]]>
</Resource>
</TestCase>
<TestCase name="testExplainWithUnion[extended=false]">
<TestCase name="testExplainWithUnion[extended=true]">
<Resource name="explain">
<![CDATA[== Abstract Syntax Tree ==
LogicalUnion(all=[true])
Expand All @@ -505,9 +505,9 @@ LogicalUnion(all=[true])
+- LogicalTableScan(table=[[default_catalog, default_database, MyTable2]])

== Optimized Physical Plan ==
Union(all=[true], union=[a, b, c])
:- BoundedStreamScan(table=[[default_catalog, default_database, MyTable1]], fields=[a, b, c])
+- BoundedStreamScan(table=[[default_catalog, default_database, MyTable2]], fields=[d, e, f])
Union(all=[true], union=[a, b, c]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
:- BoundedStreamScan(table=[[default_catalog, default_database, MyTable1]], fields=[a, b, c]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}
+- BoundedStreamScan(table=[[default_catalog, default_database, MyTable2]], fields=[d, e, f]): rowcount = , cumulative cost = {rows, cpu, io, network, memory}

== Optimized Execution Plan ==
Union(all=[true], union=[a, b, c])
Expand Down
Loading