Skip to content

Commit

Permalink
[KYUUBI #5937] PVM cause cache table not work
Browse files Browse the repository at this point in the history
# 🔍 Description
## Issue References 🔗

This pull request fixes #5937

## Describe Your Solution 🔧

If we cache a table with persist view in the query, since cache table use analyzed plan, so in kyuubi authz we will use PVM to wrap the view, but cache table use canonicalized plan, so we need to implement the `doCanonicalize()` method to ignore the impact of PVM, or it will cache cached table can't be matched.

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #5982 from AngersZhuuuu/KYUUBI-5937.

Closes #5937

e28275f [Angerszhuuuu] Update PermanentViewMarker.scala
c504103 [Angerszhuuuu] Update PermanentViewMarker.scala
19102ff [Angerszhuuuu] [KYUUBI-5937][Bug] PVM cause cache table not work

Authored-by: Angerszhuuuu <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
  • Loading branch information
AngersZhuuuu authored and pan3793 committed Jan 18, 2024
1 parent 3b2e674 commit d3a3853
Showing 1 changed file with 17 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,14 @@ import org.apache.spark.sql.catalyst.analysis.MultiInstanceRelation
import org.apache.spark.sql.catalyst.catalog.CatalogTable
import org.apache.spark.sql.catalyst.expressions.{Alias, Attribute, Cast}
import org.apache.spark.sql.catalyst.plans.QueryPlan
import org.apache.spark.sql.catalyst.plans.logical.{LeafNode, LogicalPlan, Project, Statistics}
import org.apache.spark.sql.catalyst.plans.logical.{LeafNode, LogicalPlan, Project, Statistics, View}
import org.apache.spark.sql.catalyst.trees.TreeNodeTag

case class PermanentViewMarker(child: LogicalPlan, catalogTable: CatalogTable)
extends LeafNode with MultiInstanceRelation {

private val PVM_NEW_INSTANCE_TAG = TreeNodeTag[Unit]("__PVM_NEW_INSTANCE_TAG")

override def output: Seq[Attribute] = child.output

override def argString(maxFields: Int): String = ""
Expand All @@ -38,6 +41,18 @@ case class PermanentViewMarker(child: LogicalPlan, catalogTable: CatalogTable)
val projectList = child.output.map { case attr =>
Alias(Cast(attr, attr.dataType), attr.name)(explicitMetadata = Some(attr.metadata))
}
this.copy(child = Project(projectList, child), catalogTable = catalogTable)
val newProj = Project(projectList, child)
newProj.setTagValue(PVM_NEW_INSTANCE_TAG, ())

this.copy(child = newProj, catalogTable = catalogTable)
}

override def doCanonicalize(): LogicalPlan = {
child match {
case p @ Project(_, view: View) if p.getTagValue(PVM_NEW_INSTANCE_TAG).contains(true) =>
view.canonicalized
case _ =>
child.canonicalized
}
}
}

0 comments on commit d3a3853

Please sign in to comment.