[KYUUBI #6129] [BUILD] Remove the extension spark jars before build t…

…o avoid include unspecified version

# 🔍 Description
## Issue References 🔗

This pull request fixes #5836 (comment)

## Describe Your Solution 🔧

Remove the remain extension spark jars in related target dir before start to package.

Aims to avoid include unspecified version.

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

Before this PR:
```shel
./build/dist -Pspark-3.1 ...
# your will get expected package, then you run the following command
./build/dist -Pspark-3.3
# you will find the final package include the spark 3.1 extension jar, which is un-expected.
```

#### Behavior With This Pull Request 🎉

After this PR, will clean up those extension spark jars before build to ensure we won't package the unspecified extension jar.

#### Related Unit Tests

None
---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6129 from zwangsheng/KYUUBI/clean_up_jasrs.

Closes #6129

3e5f434 [Cheng Pan] Update build/dist
d824491 [zwangsheng] Remove support spark 3.1
c83b79a [zwangsheng] [BUILD] Remove the extension spark jars before build

Lead-authored-by: zwangsheng <[email protected]>
Co-authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>

build/dist

@@ -233,6 +233,13 @@ if [[ "$HIVE_PROVIDED" == "true" ]]; then
   MVN_DIST_OPT="$MVN_DIST_OPT -Phive-provided"
 fi
 
+# clean up all versions of Spark extension, to avoid collecting outdated artifacts
+SPARK_EXTENSION_VERSIONS=('3-2' '3-3' '3-4' '3-5')
+# shellcheck disable=SC2068
+for SPARK_EXTENSION_VERSION in ${SPARK_EXTENSION_VERSIONS[@]}; do
+  rm -f "$KYUUBI_HOME/extensions/spark/kyuubi-extension-spark-$SPARK_EXTENSION_VERSION/target/kyuubi-extension-spark-${SPARK_EXTENSION_VERSION}_${SCALA_VERSION}-${VERSION}.jar"
+done
+
 export MAVEN_OPTS="${MAVEN_OPTS:--Xmx2g}"
 
 BUILD_COMMAND=("$MVN" clean install $MVN_DIST_OPT $@)
@@ -331,7 +338,6 @@ for jar in $(ls "$DISTDIR/jars/"); do
 done
 
 # Copy Kyuubi Spark extension
-SPARK_EXTENSION_VERSIONS=('3-1' '3-2' '3-3' '3-4' '3-5')
 # shellcheck disable=SC2068
 for SPARK_EXTENSION_VERSION in ${SPARK_EXTENSION_VERSIONS[@]}; do
   if [[ -f $"$KYUUBI_HOME/extensions/spark/kyuubi-extension-spark-$SPARK_EXTENSION_VERSION/target/kyuubi-extension-spark-${SPARK_EXTENSION_VERSION}_${SCALA_VERSION}-${VERSION}.jar" ]]; then