Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[KYUUBI #5535][AUTHZ] Support vacuum table command for Delta Lake
### _Why are the changes needed?_ To close #5535 . Support vacuum table command for Delta Lake. https://docs.delta.io/latest/delta-utility.html#remove-files-no-longer-referenced-by-a-delta-table == Analyzed Logical Plan == Delta Lake 3.0/Spark 3.5 ``` == Analyzed Logical Plan == path: string VacuumTableCommand false +- ResolvedTable org.apache.spark.sql.delta.catalog.DeltaCatalog3947bad4, delta_ns.table1_delta, DeltaTableV2(org.apache.spark.sql.SparkSession2e7075d4,file:/var/folders/gc/c__qhntd7s502txfp0ltxh880000gn/T/spark-warehouse-ba741e85-4c66-46f7-b1a7-10a1f32cd20c/delta_ns.db/table1_delta,Some(CatalogTable( Catalog: spark_catalog Database: delta_ns Table: table1_delta Owner: default_table_owner Created Time: Thu Nov 09 10:45:20 CST 2023 Last Access: UNKNOWN Created By: Spark 3.5.0 Type: MANAGED Provider: delta Location: file:/var/folders/gc/c__qhntd7s502txfp0ltxh880000gn/T/spark-warehouse-ba741e85-4c66-46f7-b1a7-10a1f32cd20c/delta_ns.db/table1_delta Serde Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe InputFormat: org.apache.hadoop.mapred.SequenceFileInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat Partition Provider: Catalog)),Some(delta_ns.table1_delta),None,Map()), [id#818, name#819, gender#820, birthDate#821] ``` Before Delta Lake 3.0/Spark 3.5 ``` == Analyzed Logical Plan == path: string VacuumTableCommand `delta_ns`.`table1_delta`, false ``` ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request ### _Was this patch authored or co-authored using generative AI tooling?_ No. Closes #5655 from zml1206/KYUUBI-5535. Closes #5535 9500ee7 [zml1206] Support vacuum table command for Delta Lake Authored-by: zml1206 <[email protected]> Signed-off-by: Kent Yao <[email protected]>
- Loading branch information