Skip to content

Commit d25ad1a

Browse files
committed
[KYUUBI #6287] Avoid missing matche
# 🔍 Description ## Issue References 🔗 This pull request fixes # ## Describe Your Solution 🔧 Avoid matching error caused by failed stage. error detail: ``` 24/04/09 02:11:41 ERROR AsyncEventQueue: Listener SQLOperationListener threw an exception scala.MatchError: failed (of class java.lang.String) at org.apache.spark.kyuubi.SQLOperationListener.onStageCompleted(SQLOperationListener.scala:147) at org.apache.spark.scheduler.SparkListenerBus.doPostEvent(SparkListenerBus.scala:35) at org.apache.spark.scheduler.SparkListenerBus.doPostEvent$(SparkListenerBus.scala:28) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37) at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37) at org.apache.spark.util.ListenerBus.postToAll(ListenerBus.scala:117) at org.apache.spark.util.ListenerBus.postToAll$(ListenerBus.scala:101) at org.apache.spark.scheduler.AsyncEventQueue.super$postToAll(AsyncEventQueue.scala:105) at org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:105) at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62) at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:100) at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:96) at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1387) at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:96) ``` ## Types of changes 🔖 - [X] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan 🧪 #### Behavior Without This Pull Request ⚰️ #### Behavior With This Pull Request 🎉 #### Related Unit Tests --- # Checklist 📝 - [X] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes #6287 from wForget/hotfix2. Closes #6287 01e96f0 [wforget] Avoid missing matche Authored-by: wforget <[email protected]> Signed-off-by: wforget <[email protected]> (cherry picked from commit 117d77a) Signed-off-by: wforget <[email protected]>
1 parent 3003d72 commit d25ad1a

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/SQLOperationListener.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,7 @@ class SQLOperationListener(
147147
jobInfo.numCompleteStages.getAndIncrement()
148148
}
149149
}
150+
case _ => // do nothing, failed stage not counted to numCompleteStages
150151
}
151152
withOperationLog(super.onStageCompleted(stageCompleted))
152153
}

0 commit comments

Comments
 (0)