Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RESTful API supports killing engine forcibly #6008

Closed
wants to merge 32 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
5faa5b5
kill engine forcibly
zhaohehuhu Jan 16, 2024
f3ab9c5
new parameter added to decide whether to kill forcefully
zhaohehuhu Jan 25, 2024
d6f82ff
refactor code
zhaohehuhu Jan 30, 2024
8a65cf1
refactor code
zhaohehuhu Jan 31, 2024
f11e765
add an UT
zhaohehuhu Feb 2, 2024
51827ec
fix UT
zhaohehuhu Feb 5, 2024
070aad0
refactor
zhaohehuhu Feb 19, 2024
fb9b251
add default value for forceKill param
zhaohehuhu Feb 22, 2024
bd7bb45
refactor
zhaohehuhu Feb 23, 2024
632c56b
fix unused import
zhaohehuhu Mar 14, 2024
3ad9577
rename params to support multiple engines
zhaohehuhu Mar 15, 2024
5062206
get refId by user, sharelevel and subdomain
zhaohehuhu Mar 15, 2024
513bcdc
fix UT
zhaohehuhu Mar 15, 2024
ab31382
reformat
zhaohehuhu Mar 18, 2024
634ceb6
reformat
zhaohehuhu Mar 19, 2024
ba57c2c
refactor code to delete the node and then kill application
zhaohehuhu Mar 22, 2024
11106d7
comments
turboFei Mar 26, 2024
9bacc2c
fix UTs
zhaohehuhu Apr 17, 2024
936a54e
register app mgr info
turboFei May 5, 2024
ae24ea7
refactor UT
zhaohehuhu Jun 24, 2024
3a2f597
refactor
zhaohehuhu Jun 25, 2024
a13466e
update doc and log string encoded
zhaohehuhu Jun 25, 2024
f826d05
reformat
zhaohehuhu Jun 25, 2024
0cdeede
restore ENGINE_SPARK_REGISTER_ATTRIBUTES
zhaohehuhu Jun 25, 2024
b013194
move the position of log
zhaohehuhu Jun 25, 2024
6d5d087
Update kyuubi-server/src/main/scala/org/apache/kyuubi/engine/Applicat…
pan3793 Jun 25, 2024
72d7df3
Update kyuubi-server/src/main/scala/org/apache/kyuubi/engine/Applicat…
pan3793 Jun 25, 2024
5e1b6a1
Update kyuubi-server/src/test/scala/org/apache/kyuubi/server/api/v1/A…
pan3793 Jun 25, 2024
cd5129d
fix ut
pan3793 Jun 26, 2024
efc7587
client
pan3793 Jun 27, 2024
8721a2d
log
pan3793 Jun 27, 2024
00c208a
fix
pan3793 Jun 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions docs/client/rest/rest_api.md
Original file line number Diff line number Diff line change
Expand Up @@ -457,13 +457,14 @@ Delete the specified engine.

#### Request Parameters

| Name | Description | Type |
|:------------------------|:------------------------------|:-----------------|
| type | the engine type | String(optional) |
| sharelevel | the engine share level | String(optional) |
| subdomain | the engine subdomain | String(optional) |
| proxyUser | the proxy user to impersonate | String(optional) |
| hive.server2.proxy.user | the proxy user to impersonate | String(optional) |
| Name | Description | Type |
|:------------------------|:------------------------------------|:------------------|
| type | the engine type | String(optional) |
| sharelevel | the engine share level | String(optional) |
| subdomain | the engine subdomain | String(optional) |
| proxyUser | the proxy user to impersonate | String(optional) |
| hive.server2.proxy.user | the proxy user to impersonate | String(optional) |
| kill | whether to kill the engine forcibly | Boolean(optional) |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's mention the default value is false

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it.


`proxyUser` is an alternative to `hive.server2.proxy.user`, and the current behavior is consistent with
`hive.server2.proxy.user`. When both parameters are set, `proxyUser` takes precedence.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3546,10 +3546,7 @@ object KyuubiConf {
.toSequence()
.createWithDefault(Seq(
"spark.driver.memory",
"spark.executor.memory",
"spark.kubernetes.context",
"spark.kubernetes.namespace",
"spark.master"))
"spark.executor.memory"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

restore

Copy link
Contributor Author

@zhaohehuhu zhaohehuhu Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Format issue. Restored.


val ENGINE_SPARK_INITIALIZE_SQL: ConfigEntry[Seq[String]] =
buildConf("kyuubi.engine.spark.initialize.sql")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ private[v1] class AdminResource extends ApiRequestContext with Logging {
@QueryParam("subdomain") subdomain: String,
@QueryParam("proxyUser") kyuubiProxyUser: String,
@QueryParam("hive.server2.proxy.user") hs2ProxyUser: String,
@QueryParam("forceKill") @DefaultValue("false") forceKill: Boolean): Response = {
@QueryParam("kill") @DefaultValue("false") kill: Boolean): Response = {
val activeProxyUser = Option(kyuubiProxyUser).getOrElse(hs2ProxyUser)
val userName = if (fe.isAdministrator(fe.getRealUser())) {
Option(activeProxyUser).getOrElse(fe.getRealUser())
Expand Down Expand Up @@ -307,7 +307,7 @@ private[v1] class AdminResource extends ApiRequestContext with Logging {
s"${e.getMessage}")
}

if (forceKill && engineRefId != null) {
if (kill && engineRefId != null) {
val appMgrInfo =
engineNode.attributes.get(KyuubiReservedKeys.KYUUBI_ENGINE_APP_MGR_INFO)
.map(ApplicationManagerInfo.deserialize).getOrElse(ApplicationManagerInfo(None))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -331,7 +331,7 @@ class AdminResourceSuite extends KyuubiFunSuite with RestFrontendTestHelper {
val response = webTarget.path("api/v1/admin/engine")
.queryParam("sharelevel", "USER")
.queryParam("type", "spark_sql")
.queryParam("forceKill", "true")
.queryParam("kill", "true")
.request(MediaType.APPLICATION_JSON_TYPE)
.header(AUTHORIZATION_HEADER, HttpAuthUtils.basicAuthorizationHeader(Utils.currentUser))
.delete()
Expand Down
Loading