-
Notifications
You must be signed in to change notification settings - Fork 28.7k
[SPARK-52788][SQL] Fix error of converting binary value in BinaryType to XML #51470
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems that the change test case fails. Could you double-check it, @yaooqinn ?
[info] - binary_hex_discrete.sql *** FAILED *** (170 milliseconds)
[info] Expected "[ <birth>2018</birth>
[info] <name>[45 61 73 6F 6E]</name>
[info] <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org>
[info] </ROW>
[info] <]ROW>", but got "[<ROW>
[info] <name>[45 61 73 6F 6E]</name>
[info] <birth>2018</birth>
[info] <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org>
[info] </]ROW>" Result did not match for query #5
[info] select to_xml(named_struct('name', binary('Eason'), 'birth', 2018, 'org', binary('Kindergarten Cop'))) (ThriftServerQueryTestSuite.scala:237)
Thank you @dongjoon-hyun, I filed #51488 for the test failure |
### What changes were proposed in this pull request? This PR fixes ThriftServerQueryTestSuite result comparison. When re-reading the Golden Files, if the result lines exceed the row size, we assume they contain multiple lines for a single row. In this case, we group these lines into rows first to avoid line-by-line sorting. ### Why are the changes needed? For a multiline result of a single row, it might get malformed, for example ``` [info] Expected "[ <birth>2018</birth> [info] <name>[45 61 73 6F 6E]</name> [info] <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org> [info] </ROW> [info] <]ROW>", but got "[<ROW> [info] <name>[45 61 73 6F 6E]</name> [info] <birth>2018</birth> [info] <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org> [info] </]ROW>" ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Tested with #51470 locally ### Was this patch authored or co-authored using generative AI tooling? No Closes #51488 from yaooqinn/ThriftServerQueryTestSuite. Authored-by: Kent Yao <[email protected]> Signed-off-by: Kent Yao <[email protected]>
### What changes were proposed in this pull request? This PR fixes ThriftServerQueryTestSuite result comparison. When re-reading the Golden Files, if the result lines exceed the row size, we assume they contain multiple lines for a single row. In this case, we group these lines into rows first to avoid line-by-line sorting. ### Why are the changes needed? For a multiline result of a single row, it might get malformed, for example ``` [info] Expected "[ <birth>2018</birth> [info] <name>[45 61 73 6F 6E]</name> [info] <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org> [info] </ROW> [info] <]ROW>", but got "[<ROW> [info] <name>[45 61 73 6F 6E]</name> [info] <birth>2018</birth> [info] <org>[4B 69 6E 64 65 72 67 61 72 74 65 6E 20 43 6F 70]</org> [info] </]ROW>" ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Tested with #51470 locally ### Was this patch authored or co-authored using generative AI tooling? No Closes #51488 from yaooqinn/ThriftServerQueryTestSuite. Authored-by: Kent Yao <[email protected]> Signed-off-by: Kent Yao <[email protected]> (cherry picked from commit 9565d16) Signed-off-by: Kent Yao <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, LGTM.
Merged to master. Please make a backporting PR to make it sure it pass the CI in Thank you, @yaooqinn and @peter-toth . |
What changes were proposed in this pull request?
Fix errors like 'Failed to convert value [B@2fb2f244 (class of class [B) in type BinaryType to XML'
Why are the changes needed?
bugfix
Does this PR introduce any user-facing change?
no
How was this patch tested?
new tests
Was this patch authored or co-authored using generative AI tooling?
no