[FLINK-37914][table] Add built-in OBJECT_UPDATE function #26806

raminqaf · 2025-07-17T11:50:23Z

What is the purpose of the change

This pull request implements the OBJECT_UPDATE built-in function as part of FLIP-520: Simplify StructuredType handling. The function allows users to update existing fields in structured objects by providing key-value pairs, enabling mutation operations on structured types in both SQL and Table API without requiring custom UDFs.

Brief change log

Added ObjectUpdateInputTypeStrategy for validating input arguments (structured object + key-value pairs)
Added ObjectUpdateTypeStrategy for inferring return types (same as input structured type)
Implemented ObjectUpdateFunction runtime function for performing field updates
Added OBJECT_UPDATE to BuiltInFunctionDefinitions with proper type inference strategies
Added Table API expression support via objectUpdate() method on expressions
Updated SQL and Table API documentation with examples and usage patterns
Added comprehensive validation for field names, types, and compatibility checking
Implemented proper error handling for non-existent fields and type mismatches

Verifying this change

This change added tests and can be verified as follows:

Added unit tests in ObjectUpdateInputTypeStrategyTest for input validation scenarios
Added integration tests in StructuredFunctionsITCase for end-to-end SQL functionality

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): yes (new built-in function)
The serializers: no
The runtime per-record code paths (performance sensitive): yes (new scalar function execution)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? yes
If yes, how is the feature documented? docs / JavaDocs (updated SQL functions documentation and comprehensive JavaDoc comments)

flinkbot · 2025-07-17T11:53:58Z

CI report:

74e133b Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

docs/data/sql_functions.yml

flink-python/pyflink/table/expression.py

.../flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/BaseExpressions.java

...in/java/org/apache/flink/table/types/inference/strategies/ObjectUpdateInputTypeStrategy.java

...rc/main/java/org/apache/flink/table/types/inference/strategies/ObjectUpdateTypeStrategy.java

...lanner/src/test/java/org/apache/flink/table/planner/functions/StructuredFunctionsITCase.java

twalthr

Last review comment. Almost there :-)

twalthr · 2025-07-24T08:21:52Z

docs/data/sql_functions.yml

+      This function takes a structured object and updates specified fields with new values.
+      The keys must be string literals that correspond to existing fields in the structured type.
+      If a key does not exist in the input object, an exception will be thrown.
+      If the value type is not compatible with the corresponding structured field type, 


update this sentence everywhere

twalthr · 2025-07-24T08:22:56Z

docs/data/sql_functions.yml

+      and values are the new values for those fields. At least one key-value pair must be provided.
+      The total number of arguments must be odd (object + pairs of key-value arguments).
+
+      The result type is the same structured type as the input, with the specified fields


Suggested change

The result type is the same structured type as the input, with the specified fields

The result type is the same structured type class, with the specified fields

twalthr · 2025-07-24T08:25:18Z

...in/java/org/apache/flink/table/types/inference/strategies/ObjectUpdateInputTypeStrategy.java

+ *   <li>Validates that key arguments are non-null string literals
+ *   <li>Ensures field names are not repeated in the key-value pairs
+ *   <li>Ensures field names are part of the structured type's attributes
+ *   <li>Ensures field values match the expected types defined in the structured type


Suggested change

* <li>Ensures field values match the expected types defined in the structured type

twalthr · 2025-07-24T08:27:54Z

...rc/main/java/org/apache/flink/table/types/inference/strategies/ObjectUpdateTypeStrategy.java

+ *
+ * <ul>
+ *   <li>Extracting the field definitions from the input structured type
+ *   <li>Resolving the class from the structured type


Suggested change

* <li>Resolving the class from the structured type

twalthr · 2025-07-24T10:35:42Z

...lanner/src/test/java/org/apache/flink/table/planner/functions/StructuredFunctionsITCase.java

+                                        DataTypes.FIELD("b", DataTypes.CHAR(5).notNull())))
+                        // Test update field to null
+                        .testResult(
+                                objectOf(Type1.class, "a", 42, "b", "Bob")


add a test where the object itself is null, e.g. OBJECT_UPDATED(CAST(NULL AS STRUCTURED<>), ...)

twalthr · 2025-07-24T10:36:13Z

...time/src/main/java/org/apache/flink/table/runtime/functions/scalar/ObjectUpdateFunction.java

+    }
+
+    public RowData eval(RowData rowData, Object... fieldNameAndValuePairs) {
+        GenericRowData updatedRow = (GenericRowData) rowData;


return null if rowData is null.

raminqaf force-pushed the FLINK-37914 branch from 0e8a42a to 58904bb Compare July 17, 2025 11:54

davidradl reviewed Jul 17, 2025

View reviewed changes

docs/data/sql_functions.yml Show resolved Hide resolved

github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 17, 2025

raminqaf force-pushed the FLINK-37914 branch 4 times, most recently from 56a3f1e to e437f51 Compare July 18, 2025 08:19

github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 19, 2025

raminqaf force-pushed the FLINK-37914 branch from e437f51 to 3cf5551 Compare July 21, 2025 07:10

github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 22, 2025

twalthr reviewed Jul 22, 2025

View reviewed changes

github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 22, 2025

raminqaf force-pushed the FLINK-37914 branch 3 times, most recently from 5a1cb17 to 336f2b3 Compare July 23, 2025 11:39

github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 24, 2025

[FLINK-37914][table] Add built-in OBJECT_UPDATE function

7d40760

[FLINK-37914][table] Address feedbacks

74e133b

raminqaf force-pushed the FLINK-37914 branch from 336f2b3 to 74e133b Compare July 24, 2025 06:54

github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 24, 2025

twalthr reviewed Jul 24, 2025

View reviewed changes

github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[FLINK-37914][table] Add built-in OBJECT_UPDATE function #26806

[FLINK-37914][table] Add built-in OBJECT_UPDATE function #26806

raminqaf commented Jul 17, 2025

Uh oh!

flinkbot commented Jul 17, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

twalthr left a comment

Uh oh!

twalthr Jul 24, 2025

Uh oh!

twalthr Jul 24, 2025

Uh oh!

twalthr Jul 24, 2025

Uh oh!

twalthr Jul 24, 2025

Uh oh!

twalthr Jul 24, 2025

Uh oh!

twalthr Jul 24, 2025

Uh oh!

Uh oh!

	The result type is the same structured type as the input, with the specified fields
	The result type is the same structured type class, with the specified fields

[FLINK-37914][table] Add built-in OBJECT_UPDATE function #26806

Are you sure you want to change the base?

[FLINK-37914][table] Add built-in OBJECT_UPDATE function #26806

Conversation

raminqaf commented Jul 17, 2025

What is the purpose of the change

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

Uh oh!

flinkbot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI report:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

twalthr left a comment

Choose a reason for hiding this comment

Uh oh!

twalthr Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

twalthr Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

twalthr Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

twalthr Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

twalthr Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

twalthr Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

flinkbot commented Jul 17, 2025 •

edited

Loading