[KYUUBI #5513][BATCH] Always redirect delete batch request to Kyuubi instance that owns batch session #5514
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… enabled v2 and current kyuubi not submitted this batch
zwangsheng
changed the title
pan3793
reviewed
Oct 24, 2023
| } | ||
| } else if (batchV2Enabled(metadata.requestConf) && metadata.state == "INITIALIZED" && | ||
| batchService.get.cancelUnscheduledBatch(batchId)) { | ||
| new CloseBatchResponse(true, s"Unscheduled batch $batchId is canceled.") | ||
| } else if (metadata.kyuubiInstance != fe.connectionUrl) { |
There was a problem hiding this comment.
should we add condition metadata.kyuubiInstance != null?
There was a problem hiding this comment.
Right, metadata.kyuubiInstance = null may happen when this batch picked within this function. Due to we won't fresh batch state by getting newest metadata.
We can call this delete function again to handle this case
pan3793
changed the title
pan3793
reviewed
Oct 25, 2023
kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala
Outdated
Show resolved
Hide resolved
pan3793
reviewed
Oct 25, 2023
kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala
Outdated
Show resolved
Hide resolved
…atchesResource.scala Co-authored-by: Cheng Pan <[email protected]>
…atchesResource.scala Co-authored-by: Cheng Pan <[email protected]>
pan3793
reviewed
Oct 25, 2023
kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala
Outdated
Show resolved
Hide resolved
…atchesResource.scala
pan3793
reviewed
Oct 25, 2023
kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala
Outdated
Show resolved
Hide resolved
…atchesResource.scala
pan3793
approved these changes
Oct 25, 2023
pan3793
added a commit
that referenced
this pull request
Oct 25, 2023
…instance that owns batch session ### _Why are the changes needed?_ Fixed an issue where spark submit(BatchJobSubmission) could not be closed when invoking the delete interface to delete a batch in the process of submission with batch v2 feature. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request - [x] Run exist unit test. ### _Was this patch authored or co-authored using generative AI tooling?_ No Closes #5514 from zwangsheng/KYUUBI#5513. Closes #5513 868c026 [Cheng Pan] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala 1e44acf [Cheng Pan] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala f0b3a53 [Binjie Yang] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala a238e45 [Binjie Yang] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala 565b938 [zwangsheng] modify comments 86ef016 [zwangsheng] add comment and catch spec case 4c40863 [zwangsheng] fix comments 73f89f0 [zwangsheng] [KYUUBI #5513][Improvement][BATCH] Redirect delete batch request when enabled v2 and current kyuubi not submitted this batch Lead-authored-by: zwangsheng <[email protected]> Co-authored-by: Cheng Pan <[email protected]> Co-authored-by: Binjie Yang <[email protected]> Signed-off-by: Cheng Pan <[email protected]> (cherry picked from commit feb9d16) Signed-off-by: Cheng Pan <[email protected]>
|
Merged to master/1.8 |
davidyuan1223
pushed a commit
to davidyuan1223/kyuubi
that referenced
this pull request
Oct 26, 2023
…yuubi instance that owns batch session ### _Why are the changes needed?_ Fixed an issue where spark submit(BatchJobSubmission) could not be closed when invoking the delete interface to delete a batch in the process of submission with batch v2 feature. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request - [x] Run exist unit test. ### _Was this patch authored or co-authored using generative AI tooling?_ No Closes apache#5514 from zwangsheng/KYUUBI#5513. Closes apache#5513 868c026 [Cheng Pan] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala 1e44acf [Cheng Pan] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala f0b3a53 [Binjie Yang] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala a238e45 [Binjie Yang] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala 565b938 [zwangsheng] modify comments 86ef016 [zwangsheng] add comment and catch spec case 4c40863 [zwangsheng] fix comments 73f89f0 [zwangsheng] [KYUUBI apache#5513][Improvement][BATCH] Redirect delete batch request when enabled v2 and current kyuubi not submitted this batch Lead-authored-by: zwangsheng <[email protected]> Co-authored-by: Cheng Pan <[email protected]> Co-authored-by: Binjie Yang <[email protected]> Signed-off-by: Cheng Pan <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
Fixed an issue where spark submit(BatchJobSubmission) could not be closed when invoking the delete interface to delete a batch in the process of submission with batch v2 feature.
How was this patch tested?
Add some test cases that check the changes thoroughly including negative and positive cases if possible
Add screenshots for manual tests if appropriate
Run test locally before make a pull request
Run exist unit test.
Was this patch authored or co-authored using generative AI tooling?
No