Skip to content

[SPARK-52788][SQL] Fix error of converting binary value in BinaryType to XML #51470

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

yaooqinn
Copy link
Member

What changes were proposed in this pull request?

Fix errors like 'Failed to convert value [B@2fb2f244 (class of class [B) in type BinaryType to XML'

Why are the changes needed?

bugfix

Does this PR introduce any user-facing change?

no

How was this patch tested?

new tests

Was this patch authored or co-authored using generative AI tooling?

no

@github-actions github-actions bot added the SQL label Jul 14, 2025
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the change test case fails. Could you double-check it, @yaooqinn ?

[info] - binary_hex_discrete.sql *** FAILED *** (170 milliseconds)
[info]   Expected "[    <birth>2018</birth>
[info]       <name>[45 61 73 6F 6E]</name>
[info]       <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org>
[info]   </ROW>
[info]   <]ROW>", but got "[<ROW>
[info]       <name>[45 61 73 6F 6E]</name>
[info]       <birth>2018</birth>
[info]       <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org>
[info]   </]ROW>" Result did not match for query #5
[info]   select to_xml(named_struct('name', binary('Eason'), 'birth', 2018, 'org', binary('Kindergarten Cop'))) (ThriftServerQueryTestSuite.scala:237)

@yaooqinn
Copy link
Member Author

Thank you @dongjoon-hyun, I filed #51488 for the test failure

yaooqinn added a commit that referenced this pull request Jul 15, 2025
### What changes were proposed in this pull request?

This PR fixes ThriftServerQueryTestSuite result comparison. When re-reading the Golden Files, if the result lines exceed the row size, we assume they contain multiple lines for a single row. In this case, we group these lines into rows first to avoid line-by-line sorting.

### Why are the changes needed?

For a multiline result of a single row, it might get malformed, for example

```
[info]   Expected "[    <birth>2018</birth>
[info]       <name>[45 61 73 6F 6E]</name>
[info]       <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org>
[info]   </ROW>
[info]   <]ROW>", but got "[<ROW>
[info]       <name>[45 61 73 6F 6E]</name>
[info]       <birth>2018</birth>
[info]       <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org>
[info]   </]ROW>"
```

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

Tested with #51470 locally

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #51488 from yaooqinn/ThriftServerQueryTestSuite.

Authored-by: Kent Yao <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
yaooqinn added a commit that referenced this pull request Jul 15, 2025
### What changes were proposed in this pull request?

This PR fixes ThriftServerQueryTestSuite result comparison. When re-reading the Golden Files, if the result lines exceed the row size, we assume they contain multiple lines for a single row. In this case, we group these lines into rows first to avoid line-by-line sorting.

### Why are the changes needed?

For a multiline result of a single row, it might get malformed, for example

```
[info]   Expected "[    <birth>2018</birth>
[info]       <name>[45 61 73 6F 6E]</name>
[info]       <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org>
[info]   </ROW>
[info]   <]ROW>", but got "[<ROW>
[info]       <name>[45 61 73 6F 6E]</name>
[info]       <birth>2018</birth>
[info]       <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org>
[info]   </]ROW>"
```

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

Tested with #51470 locally

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #51488 from yaooqinn/ThriftServerQueryTestSuite.

Authored-by: Kent Yao <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
(cherry picked from commit 9565d16)
Signed-off-by: Kent Yao <[email protected]>
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM.

@dongjoon-hyun
Copy link
Member

Merged to master. Please make a backporting PR to make it sure it pass the CI in branch-4.0, @yaooqinn .

Thank you, @yaooqinn and @peter-toth .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants