Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[KYUUBI #3444] Support the planOnly mode of kyuubi spark engine support SQL lineage #3558

Closed
wants to merge 3 commits into from

Conversation

iodone
Copy link
Contributor

@iodone iodone commented Sep 26, 2022

Closes #3444

Why are the changes needed?

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before make a pull request

Copy link
Contributor

@ulysses-you ulysses-you left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks a good feature, cc @yaooqinn @cfmcgrady

@github-actions github-actions bot added the kind:documentation Documentation is a feature! label Sep 28, 2022
@iodone iodone force-pushed the kyuubi-3444 branch 2 times, most recently from f66202d to 74a2783 Compare October 8, 2022 08:26
@codecov-commenter
Copy link

codecov-commenter commented Oct 8, 2022

Codecov Report

Merging #3558 (acaa72a) into master (c844f18) will not change coverage.
The diff coverage is 0.00%.

@@          Coverage Diff           @@
##           master   #3558   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files         589     590    +1     
  Lines       33311   33346   +35     
  Branches     4399    4404    +5     
======================================
- Misses      33311   33346   +35     
Files Changed Coverage Δ
.../kyuubi/plugin/lineage/LineageParserProvider.scala 0.00% <0.00%> (ø)
...ubi/engine/spark/operation/PlanOnlyStatement.scala 0.00% <0.00%> (ø)
...ala/org/apache/spark/kyuubi/SparkUtilsHelper.scala 0.00% <0.00%> (ø)
...in/scala/org/apache/kyuubi/config/KyuubiConf.scala 0.00% <0.00%> (ø)
...ala/org/apache/kyuubi/operation/PlanOnlyMode.scala 0.00% <0.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@iodone iodone force-pushed the kyuubi-3444 branch 2 times, most recently from df6d76d to 91bfa68 Compare October 8, 2022 12:04
@ulysses-you
Copy link
Contributor

the github action seems get sick.. every commit need approve to run : (

@iodone iodone force-pushed the kyuubi-3444 branch 6 times, most recently from 04b8e24 to af79aa7 Compare October 13, 2022 12:27
@wForget
Copy link
Member

wForget commented Dec 7, 2022

We can define a PlanOnlyStatement mode interface in Engine, and then implement it in the lineage plugin.

@iodone
Copy link
Contributor Author

iodone commented Sep 1, 2023

@yaooqinn @wForget This feature has been overlooked. A new version has recently been released. The engine has removed the strong dependency on the plugin by reflecting on the provider class of the plugin. Can you help me review it again?

@pan3793
Copy link
Member

pan3793 commented Sep 1, 2023

cc @ulysses-you

@ulysses-you
Copy link
Contributor

thanks, merging to master

@ulysses-you ulysses-you added this to the v1.8.0 milestone Sep 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] The planOnly mode of kyuubi spark engine support SQL lineage output
6 participants