Skip to content

Commit

Permalink
[KYUUBI #6287] Avoid missing matche
Browse files Browse the repository at this point in the history
# 🔍 Description
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Avoid matching error caused by failed stage.

error detail:

```
24/04/09 02:11:41 ERROR AsyncEventQueue: Listener SQLOperationListener threw an exception
scala.MatchError: failed (of class java.lang.String)
	at org.apache.spark.kyuubi.SQLOperationListener.onStageCompleted(SQLOperationListener.scala:147)
	at org.apache.spark.scheduler.SparkListenerBus.doPostEvent(SparkListenerBus.scala:35)
	at org.apache.spark.scheduler.SparkListenerBus.doPostEvent$(SparkListenerBus.scala:28)
	at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
	at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
	at org.apache.spark.util.ListenerBus.postToAll(ListenerBus.scala:117)
	at org.apache.spark.util.ListenerBus.postToAll$(ListenerBus.scala:101)
	at org.apache.spark.scheduler.AsyncEventQueue.super$postToAll(AsyncEventQueue.scala:105)
	at org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:105)
	at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23)
	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
	at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:100)
	at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:96)
	at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1387)
	at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:96)
```

## Types of changes 🔖

- [X] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6287 from wForget/hotfix2.

Closes #6287

01e96f0 [wforget] Avoid missing matche

Authored-by: wforget <[email protected]>
Signed-off-by: wforget <[email protected]>
  • Loading branch information
wForget committed Apr 11, 2024
1 parent b9c97dd commit 117d77a
Showing 1 changed file with 1 addition and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ class SQLOperationListener(
jobInfo.numCompleteStages.getAndIncrement()
}
}
case _ => // do nothing, failed stage not counted to numCompleteStages
}
val taskMetrics = stageInfo.taskMetrics
if (taskMetrics != null) {
Expand Down

0 comments on commit 117d77a

Please sign in to comment.