Skip to content

Commit

Permalink
[SPARK-51073][SQL] Remove Unstable from `SparkSessionExtensionsProv…
Browse files Browse the repository at this point in the history
…ider` trait

### What changes were proposed in this pull request?

This PR aims to remove `Unstable` from `SparkSessionExtensionsProvider` trait from Apache Spark 4.1.0.

https://github.com/apache/spark/blob/400e2b29caf43b443f2e0c42aee92c4d0e3e0eb7/sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensionsProvider.scala#L78-L81

### Why are the changes needed?

Since Apache Spark 3.2.0, we provide `SparkSessionExtensionsProvider` trait without any change to allow third-party developers to provide a resource file that contains default extensions.
- #32515

Apache Iceberg has been using this since 0.11.0.
- https://iceberg.apache.org/docs/1.7.1/spark-configuration/#sql-extensions
    > Iceberg 0.11.0 and later add an extension module to Spark to add new SQL commands, like CALL for stored procedures or ALTER TABLE ... WRITE ORDERED BY.

Although `SparkSessionExtensions` is an unstable developer API, `SparkSessionExtensionsProvider` trait itself and ServiceLoader framework is stable and it's unlikely for Apache Spark to change this `trait`.

https://github.com/apache/spark/blob/400e2b29caf43b443f2e0c42aee92c4d0e3e0eb7/sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensions.scala#L106-L109

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #49780 from dongjoon-hyun/SPARK-51073.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
dongjoon-hyun committed Feb 10, 2025
1 parent e823afa commit 222dd81
Showing 1 changed file with 1 addition and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,10 @@

package org.apache.spark.sql

import org.apache.spark.annotation.{DeveloperApi, Since, Unstable}
import org.apache.spark.annotation.{DeveloperApi, Since}

// scalastyle:off line.size.limit
/**
* :: Unstable ::
*
* Base trait for implementations used by [[SparkSessionExtensions]]
*
*
Expand Down Expand Up @@ -76,7 +74,6 @@ import org.apache.spark.annotation.{DeveloperApi, Since, Unstable}
* @since 3.2.0
*/
@DeveloperApi
@Unstable
@Since("3.2.0")
trait SparkSessionExtensionsProvider extends Function1[SparkSessionExtensions, Unit]
// scalastyle:on line.size.limit

0 comments on commit 222dd81

Please sign in to comment.