Skip to content

Commit

Permalink
[KYUUBI #5305] Return dummy log when batch job before being added to …
Browse files Browse the repository at this point in the history
…session manager

…sion manager

### _Why are the changes needed?_

As title

Current we open session first then hand this session with batch by SessionManager:
https://github.com/apache/kyuubi/blob/413683507079f8f97217e8099177f0b16bc9192d/kyuubi-common/src/main/scala/org/apache/kyuubi/session/SessionManager.scala#L97-L121

We may face `No local log `exception, when kyuubi server picked this batch job to open session but not handled by SessionManager, due to time gap and code gap.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [X] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request

### _Was this patch authored or co-authored using generative AI tooling?_

No

Closes #5305 from zwangsheng/server/improve_localLog_when_job_under_submmit.

Closes #5305

14e27c1 [Cheng Pan] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/BatchesResource.scala
a5b3ca1 [zwangsheng] [V2] Retunr dump log when batch job was picked but not handler by session manager

Lead-authored-by: zwangsheng <[email protected]>
Co-authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
  • Loading branch information
zwangsheng and pan3793 committed Sep 18, 2023
1 parent 4136835 commit ece25f2
Showing 1 changed file with 7 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -415,6 +415,13 @@ private[v1] class BatchesResource extends ApiRequestContext with Logging {
} else if (fe.connectionUrl != metadata.kyuubiInstance) {
val internalRestClient = getInternalRestClient(metadata.kyuubiInstance)
internalRestClient.getBatchLocalLog(userName, batchId, from, size)
} else if (batchV2Enabled(metadata.requestConf) &&
// in batch v2 impl, the operation state is changed from PENDING to RUNNING
// before being added to SessionManager.
(metadata.state == "PENDING" || metadata.state == "RUNNING")) {
info(s"Batch $batchId is waiting for submitting")
val dummyLogs = List(s"Batch $batchId is waiting for submitting").asJava
new OperationLog(dummyLogs, dummyLogs.size)
} else {
throw new NotFoundException(s"No local log found for batch: $batchId")
}
Expand Down

0 comments on commit ece25f2

Please sign in to comment.