Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump Kyuubi Shaded 0.3.0 #6117

Closed
wants to merge 8 commits into from
Closed

Bump Kyuubi Shaded 0.3.0 #6117

wants to merge 8 commits into from

Conversation

pan3793
Copy link
Member

@pan3793 pan3793 commented Feb 29, 2024

🔍 Description

Issue References 🔗

This PR upgrades Kyuubi Shaded to 0.3.0, the major changes are:

  • upgrade Thrift from 0.9.3-1 to 0.16.0
  • upgrade JLine used by Zookeeper CLI from 0.9.94 to 2.14.6
  • split kyuubi-relocated-thrift from kyuubi-relocated-hive-service-rpc
  • introduce a light kyuubi-relocated-hive-metastore-client for getting DelegationToken

Due to THRIFT-5297 (fixed in 0.14.0) removes requestTimeout and beBackoffSlotLength, the following Kyuubi configurations are removed too.

  • kyuubi.frontend.login.timeout
  • kyuubi.frontend.thrift.login.timeout
  • kyuubi.frontend.backoff.slot.length
  • kyuubi.frontend.thrift.backoff.slot.length

Previously, Zookeeper 3.4 depends on JLine 0.9.94 while Zookeeper 3.6 and Hive Beeline depends on JLine 2.14.6, we pulls different versions of JLine into the binary distribution. Now we align them to a single JLine 2.14.6.

Describe Your Solution 🔧

Upgrade dependencies and change code to adapt to the breaking changes, also upgrade migration guide to mention the removed configurations.

Types of changes 🔖

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Pass GA.


Checklist 📝

Be nice. Be informative.

@codecov-commenter
Copy link

codecov-commenter commented Feb 29, 2024

Codecov Report

Attention: Patch coverage is 44.44444% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 61.01%. Comparing base (85deecc) to head (241fae4).
Report is 2 commits behind head on master.

❗ Current head 241fae4 differs from pull request most recent head 5639c9f. Consider uploading reports for the commit 5639c9f to get more accurate results

Files Patch % Lines
...apache/kyuubi/jdbc/hive/auth/TFilterTransport.java 40.00% 3 Missing ⚠️
...a/org/apache/kyuubi/service/TFrontendService.scala 0.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #6117      +/-   ##
============================================
- Coverage     61.13%   61.01%   -0.13%     
+ Complexity       24       23       -1     
============================================
  Files           625      623       -2     
  Lines         37263    37177      -86     
  Branches       5046     5039       -7     
============================================
- Hits          22781    22683      -98     
- Misses        12020    12035      +15     
+ Partials       2462     2459       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions github-actions bot added the kind:documentation Documentation is a feature! label Feb 29, 2024
@pan3793 pan3793 changed the title [WIP] Bump Kyuubi Shaded 0.3.0 Bump Kyuubi Shaded 0.3.0 Mar 7, 2024
@pan3793 pan3793 marked this pull request as ready for review March 7, 2024 13:39
@pan3793
Copy link
Member Author

pan3793 commented Mar 7, 2024

@SteNicholas after two years, I'd like to re-attempt this upgrading to address CVEs #1953

@pan3793 pan3793 self-assigned this Mar 7, 2024
@pan3793 pan3793 added this to the v1.9.0 milestone Mar 7, 2024
@pan3793
Copy link
Member Author

pan3793 commented Mar 7, 2024

also cc @zhouyifan279

@zhouyifan279
Copy link
Contributor

LGTM

errMsg.contains(s"Socket for ${SessionHandle(handle)} is closed"))
errMsg.contains(s"Socket for ${SessionHandle(handle)} is closed") ||
errMsg.contains("Socket is closed by peer") ||
errMsg.contains("SparkContext was shut down"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the errMsg contain Broken pipe (Write failed)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think no, because the error is likely occurs on reading

@pan3793
Copy link
Member Author

pan3793 commented Mar 8, 2024

Thanks, merging to master

@pan3793 pan3793 closed this in 182f434 Mar 8, 2024
zhouyifan279 pushed a commit to zhouyifan279/kyuubi that referenced this pull request Mar 11, 2024
# 🔍 Description
## Issue References 🔗

This PR upgrades Kyuubi Shaded to 0.3.0, the major changes are:

- upgrade Thrift from 0.9.3-1 to 0.16.0
- upgrade JLine used by Zookeeper CLI from 0.9.94 to 2.14.6
- split kyuubi-relocated-thrift from kyuubi-relocated-hive-service-rpc
- introduce a light kyuubi-relocated-hive-metastore-client for getting DelegationToken

Due to THRIFT-5297 (fixed in 0.14.0) removes requestTimeout and beBackoffSlotLength, the following Kyuubi configurations are removed too.

- kyuubi.frontend.login.timeout
- kyuubi.frontend.thrift.login.timeout
- kyuubi.frontend.backoff.slot.length
- kyuubi.frontend.thrift.backoff.slot.length

Previously, Zookeeper 3.4 depends on JLine 0.9.94 while Zookeeper 3.6 and Hive Beeline depends on JLine 2.14.6, we pulls different versions of JLine into the binary distribution. Now we align them to a single JLine 2.14.6.

## Describe Your Solution 🔧

Upgrade dependencies and change code to adapt to the breaking changes, also upgrade migration guide to mention the removed configurations.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Pass GA.

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes apache#6117 from pan3793/shaded-0.3.

Closes apache#6117

5639c9f [Cheng Pan] nit
d281cdb [Cheng Pan] fix
184e1b9 [Cheng Pan] fix
1628337 [Cheng Pan] remove unused conf
24db2d5 [Cheng Pan] try fix
1e995bb [Cheng Pan] nit
1d8e9bc [Cheng Pan] dep list
7f33624 [Cheng Pan] Bump Kyuubi Shaded 0.3.0

Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
zhaohehuhu pushed a commit to zhaohehuhu/incubator-kyuubi that referenced this pull request Mar 21, 2024
# 🔍 Description
## Issue References 🔗

This PR upgrades Kyuubi Shaded to 0.3.0, the major changes are:

- upgrade Thrift from 0.9.3-1 to 0.16.0
- upgrade JLine used by Zookeeper CLI from 0.9.94 to 2.14.6
- split kyuubi-relocated-thrift from kyuubi-relocated-hive-service-rpc
- introduce a light kyuubi-relocated-hive-metastore-client for getting DelegationToken

Due to THRIFT-5297 (fixed in 0.14.0) removes requestTimeout and beBackoffSlotLength, the following Kyuubi configurations are removed too.

- kyuubi.frontend.login.timeout
- kyuubi.frontend.thrift.login.timeout
- kyuubi.frontend.backoff.slot.length
- kyuubi.frontend.thrift.backoff.slot.length

Previously, Zookeeper 3.4 depends on JLine 0.9.94 while Zookeeper 3.6 and Hive Beeline depends on JLine 2.14.6, we pulls different versions of JLine into the binary distribution. Now we align them to a single JLine 2.14.6.

## Describe Your Solution 🔧

Upgrade dependencies and change code to adapt to the breaking changes, also upgrade migration guide to mention the removed configurations.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Pass GA.

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes apache#6117 from pan3793/shaded-0.3.

Closes apache#6117

5639c9f [Cheng Pan] nit
d281cdb [Cheng Pan] fix
184e1b9 [Cheng Pan] fix
1628337 [Cheng Pan] remove unused conf
24db2d5 [Cheng Pan] try fix
1e995bb [Cheng Pan] nit
1d8e9bc [Cheng Pan] dep list
7f33624 [Cheng Pan] Bump Kyuubi Shaded 0.3.0

Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
@pan3793 pan3793 deleted the shaded-0.3 branch June 6, 2024 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants