Skip to content

[FLINK-37914][table] Add built-in OBJECT_UPDATE function #26806

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

raminqaf
Copy link
Contributor

What is the purpose of the change

This pull request implements the OBJECT_UPDATE built-in function as part of FLIP-520: Simplify StructuredType handling. The function allows users to update existing fields in structured objects by providing key-value pairs, enabling mutation operations on structured types in both SQL and Table API without requiring custom UDFs.

Brief change log

  • Added ObjectUpdateInputTypeStrategy for validating input arguments (structured object + key-value pairs)
  • Added ObjectUpdateTypeStrategy for inferring return types (same as input structured type)
  • Implemented ObjectUpdateFunction runtime function for performing field updates
  • Added OBJECT_UPDATE to BuiltInFunctionDefinitions with proper type inference strategies
  • Added Table API expression support via objectUpdate() method on expressions
  • Updated SQL and Table API documentation with examples and usage patterns
  • Added comprehensive validation for field names, types, and compatibility checking
  • Implemented proper error handling for non-existent fields and type mismatches

Verifying this change

This change added tests and can be verified as follows:

  • Added unit tests in ObjectUpdateInputTypeStrategyTest for input validation scenarios
  • Added integration tests in StructuredFunctionsITCase for end-to-end SQL functionality

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): yes (new built-in function)
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): yes (new scalar function execution)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? docs / JavaDocs (updated SQL functions documentation and comprehensive JavaDoc comments)

@flinkbot
Copy link
Collaborator

flinkbot commented Jul 17, 2025

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@github-actions github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 17, 2025
@raminqaf raminqaf force-pushed the FLINK-37914 branch 4 times, most recently from 56a3f1e to e437f51 Compare July 18, 2025 08:19
@github-actions github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 19, 2025
@github-actions github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 22, 2025
@github-actions github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 22, 2025
@raminqaf raminqaf force-pushed the FLINK-37914 branch 3 times, most recently from 5a1cb17 to 336f2b3 Compare July 23, 2025 11:39
@github-actions github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 24, 2025
@github-actions github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 24, 2025
Copy link
Contributor

@twalthr twalthr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last review comment. Almost there :-)

This function takes a structured object and updates specified fields with new values.
The keys must be string literals that correspond to existing fields in the structured type.
If a key does not exist in the input object, an exception will be thrown.
If the value type is not compatible with the corresponding structured field type,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update this sentence everywhere

and values are the new values for those fields. At least one key-value pair must be provided.
The total number of arguments must be odd (object + pairs of key-value arguments).

The result type is the same structured type as the input, with the specified fields
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The result type is the same structured type as the input, with the specified fields
The result type is the same structured type class, with the specified fields

* <li>Validates that key arguments are non-null string literals
* <li>Ensures field names are not repeated in the key-value pairs
* <li>Ensures field names are part of the structured type's attributes
* <li>Ensures field values match the expected types defined in the structured type
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* <li>Ensures field values match the expected types defined in the structured type

*
* <ul>
* <li>Extracting the field definitions from the input structured type
* <li>Resolving the class from the structured type
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* <li>Resolving the class from the structured type

DataTypes.FIELD("b", DataTypes.CHAR(5).notNull())))
// Test update field to null
.testResult(
objectOf(Type1.class, "a", 42, "b", "Bob")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a test where the object itself is null, e.g. OBJECT_UPDATED(CAST(NULL AS STRUCTURED<>), ...)

}

public RowData eval(RowData rowData, Object... fieldNameAndValuePairs) {
GenericRowData updatedRow = (GenericRowData) rowData;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return null if rowData is null.

@github-actions github-actions bot added community-reviewed PR has been reviewed by the community. and removed community-reviewed PR has been reviewed by the community. labels Jul 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community-reviewed PR has been reviewed by the community.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants