[KYUUBI #5396] SBT build support

[davidyuan1223]

🔍 Description

Issue References 🔗

This pull request fixes #5396

Describe Your Solution 🔧

This pull request used to support sbt build.

1. execute sbt

fuyuanyuan@bogon kyuubi % ./build/sbt
Using /Library/Java/JavaVirtualMachines/jdk1.8.0_333.jdk/Contents/Home as default JAVA_HOME.
Note, this will be overridden by -java-home if it is set.
sbt-launch-1.9.0.jar
[info] welcome to sbt 1.9.0 (Oracle Corporation Java 1.8.0_333)
[info] loading settings for project kyuubi-build from plugin.sbt ...
[info] loading project definition from /Users/fuyuanyuan/Programs/code/github/kyuubi/project
[info] compiling 1 Scala source to /Users/fuyuanyuan/Programs/code/github/kyuubi/project/target/scala-2.12/sbt-1.0/classes ...
[info] resolving key references (34145 settings) ...
[info] set current project to kyuubi-parent (in build file:/Users/fuyuanyuan/Programs/code/github/kyuubi/)
[info] 
[info] Here are some highlights of sbt 1.9.0:
[info]   - POM consistency of sbt plugin publishing
[info]   - sbt new, a text-based adventure
[info]   - Deprecation of IntegrationTest configuration
[info] See https://eed3si9n.com/sbt-1.9.0 for full release notes.
[info] Hide the banner for this release by running `skipBanner`.
[info] sbt server started at local:///Users/fuyuanyuan/.sbt/1.0/server/9b487a3b3f2063d363b2/sock
[info] started sbt server

2. load projects

sbt:kyuubi-parent> projects
[info] In file:/Users/fuyuanyuan/Programs/code/github/kyuubi/
[info]     integration-tests
[info]   * kyuubi
[info]     kyuubi-assembly
[info]     kyuubi-chat-engine
[info]     kyuubi-codecov
[info]     kyuubi-common
[info]     kyuubi-ctl
[info]     kyuubi-download
[info]     kyuubi-events
[info]     kyuubi-extension-spark-jdbc-dialect
[info]     kyuubi-flink-it
[info]     kyuubi-flink-sql-engine
[info]     kyuubi-ha
[info]     kyuubi-hive-beeline
[info]     kyuubi-hive-it
[info]     kyuubi-hive-jdbc
[info]     kyuubi-hive-jdbc-shaded
[info]     kyuubi-hive-sql-engine
[info]     kyuubi-jdbc-engine
[info]     kyuubi-jdbc-it
[info]     kyuubi-metrics
[info]     kyuubi-rest-client
[info]     kyuubi-server
[info]     kyuubi-server-plugin
[info]     kyuubi-spark-authz
[info]     kyuubi-spark-authz-shaded
[info]     kyuubi-spark-connector-common
[info]     kyuubi-spark-connector-tpcds
[info]     kyuubi-spark-connector-tpch
[info]     kyuubi-spark-lineage
[info]     kyuubi-spark-sql-engine
[info]     kyuubi-trino-engine
[info]     kyuubi-trino-it
[info]     kyuubi-util
[info]     kyuubi-util-scala
[info]     kyuubi-zookeeper
[info]     kyuubi-zookeeper-it

3. test module kyuubi-util to clean

sbt:kyuubi-util> project kyuubi-util
[info] set current project to kyuubi-util (in build file:/Users/fuyuanyuan/Programs/code/github/kyuubi/)
sbt:kyuubi-util> clean
[success] Total time: 0 s, completed 2023-11-26 16:24:39

4. test module kyuubi-util to package

sbt:kyuubi-util> package
[info] Running scalastyle on kyuubi-util in compile
[info] scalastyle using config /Users/fuyuanyuan/Programs/code/github/kyuubi/scalastyle-on-compile.generated.xml
[info] scalastyle Processed 0 file(s)
[info] scalastyle Found 0 errors
[info] scalastyle Found 0 warnings
[info] scalastyle Found 0 infos
[info] scalastyle Finished in 3 ms
[success] created output: /Users/fuyuanyuan/Programs/code/github/kyuubi/kyuubi-util/target
[info] compiling 4 Java sources to /Users/fuyuanyuan/Programs/code/github/kyuubi/kyuubi-util/target/scala-2.12/classes ...
[success] Total time: 2 s, completed 2023-11-26 16:24:54

The load projects are base projects, if we want load another projects, we need use -P${profile} , for examples we use ./build/sbt -Pspark-3.3 projects , the projects will contains [info] kyuubi-extension-spark-3-3

In the end, this pull request is a basic support for sbt, some speical dependency in project, we need change the Build.scala to support.

Types of changes 🔖

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Behavior Without This Pull Request ⚰️

Behavior With This Pull Request 🎉

Related Unit Tests

---

Checklists

📝 Author Self Checklist

• My code follows the style guidelines of this project
• I have performed a self-review
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• This patch was not authored or co-authored using Generative Tooling

📝 Committer Pre-Merge Checklist

• Pull request title is okay.
• No license issues.
• Milestone correctly set?
• Test coverage is ok
• Assignees are selected.
• Minimum number of approvals
• No changes are requested

Be nice. Be informative.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (5481bf5) 61.37% compared to head (c833c68) 61.37%.
Report is 5 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #5775      +/-   ##
============================================
- Coverage     61.37%   61.37%   -0.01%     
  Complexity       23       23              
============================================
  Files           607      607              
  Lines         35927    35944      +17     
  Branches       4931     4936       +5     
============================================
+ Hits          22051    22061      +10     
+ Misses        11492    11490       -2     
- Partials       2384     2393       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[davidyuan1223]

What do you think about this code change? Please give me some advice, i will fix them, then add the doc about sbt.
@pan3793 @cfmcgrady

[davidyuan1223]

What do you think about this code change? Please give me some advice, i will fix them, then add the doc about sbt. @pan3793 @cfmcgrady

In addition to this, where sbt ci should be added to github workflow, and sbt may not support all projects at this time

[davidyuan1223]

@pan3793 @cfmcgrady @bowenliang123 Hello, I'm sorry, I don't think I can complete this issue. Although I have set up a basic environment, during local debugging, I always fail when I test these projects. After searching a lot of solutions, I still can't solve these problems, so I think this issue may be more suitable for people who are proficient in SBT to complete.

[pan3793]

@davidyuan1223 this may be challenging for developers who are not familiar with SBT, anyway, thanks for your effort

[bowenliang123]

This's quite challenging to bring a complicated multi-module Maven project into sbt native project. It requires experience, insight and fine tuning in both tools and related counterpart plugins, even with the help of sbt maven migration kits.
Thanks for the attempts and efforts.