Skip to content

Commit ff7b4a4

Browse files
[SPARK-51315][SQL] Enabling object level collations by default
### What changes were proposed in this pull request? Enabling the flag for object level collations by default, as the feature is now working end-to-end. Also, a minor fix is made in DDL command resolution for AlterTableCommand, ensuring that the altered object is an instance of ResolvedTable, to avoid exposing this path for other types such as ResolvedPersistentView. This is just additional safety mechanism. ### Why are the changes needed? The feature is now working end-to-end, and can be enabled accordingly. ### Does this PR introduce _any_ user-facing change? No. Previous PRs related to this functionality already introduced all underlying user facing changes. ### How was this patch tested? Existing dedicated tests already cover this functionality (the flag was already enabled in testing environment). ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#50082 from dejankrak-db/enable-object-level-collations. Lead-authored-by: Dejan Krakovic <[email protected]> Co-authored-by: Wenchen Fan <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent 971b9a4 commit ff7b4a4

File tree

2 files changed

+13
-8
lines changed

2 files changed

+13
-8
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveDDLCommandStringTypes.scala

+12-7
Original file line numberDiff line numberDiff line change
@@ -45,17 +45,22 @@ object ResolveDDLCommandStringTypes extends Rule[LogicalPlan] {
4545
table match {
4646
case createTable: CreateTable if createTable.tableSpec.collation.isDefined =>
4747
StringType(createTable.tableSpec.collation.get)
48+
4849
case createView: CreateView if createView.collation.isDefined =>
4950
StringType(createView.collation.get)
51+
5052
case alterTable: AlterTableCommand if alterTable.table.resolved =>
51-
val collation = Option(alterTable
52-
.table.asInstanceOf[ResolvedTable]
53-
.table.properties.get(TableCatalog.PROP_COLLATION))
54-
if (collation.isDefined) {
55-
StringType(collation.get)
56-
} else {
57-
StringType(defaultCollation)
53+
alterTable.table match {
54+
case resolvedTbl: ResolvedTable =>
55+
val collation = resolvedTbl.table.properties.getOrDefault(
56+
TableCatalog.PROP_COLLATION, defaultCollation)
57+
StringType(collation)
58+
59+
case _ =>
60+
// As a safeguard, use the default collation for unknown cases.
61+
StringType(defaultCollation)
5862
}
63+
5964
case _ => StringType(defaultCollation)
6065
}
6166
}

sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala

+1-1
Original file line numberDiff line numberDiff line change
@@ -868,7 +868,7 @@ object SQLConf {
868868
)
869869
.version("4.0.0")
870870
.booleanConf
871-
.createWithDefault(Utils.isTesting)
871+
.createWithDefault(true)
872872

873873
lazy val TRIM_COLLATION_ENABLED =
874874
buildConf("spark.sql.collation.trim.enabled")

0 commit comments

Comments
 (0)