Skip to content

Files

This branch is 406 commits behind apache/hudi:master.

rfc

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
Mar 3, 2022
Apr 9, 2022
Feb 1, 2022
Mar 2, 2022
Nov 25, 2021
Jan 28, 2022
Jan 26, 2023
Mar 11, 2022
May 2, 2022
Apr 1, 2022
Jan 18, 2023
Mar 4, 2022
Feb 14, 2023
Apr 1, 2022
May 18, 2022
Jan 30, 2023
Jun 14, 2022
Jul 21, 2022
Feb 17, 2023
Aug 28, 2022
Mar 4, 2024
Feb 9, 2024
Dec 8, 2023
Dec 21, 2023
May 30, 2023
Jul 11, 2023
Sep 6, 2023
Mar 11, 2024
Sep 28, 2023
Aug 2, 2024
Sep 25, 2024
Jan 12, 2023

RFCs

The RFC process is documented on our site. Please familiarize yourself with it, before working a new RFC.

Status can be one of these values.

Status Meaning
UNDER REVIEW RFC has been proposed and community is actively debating the design/proposal.
IN PROGRESS The initial phase of implementation is underway.
ONGOING Some or most work has landed; community continues to improve or build follow on phases.
ABANDONED The proposal was not implemented, due to various reasons.
COMPLETED All work is deemed complete.

The list of all RFCs can be found here.

Older RFC content is still here.

RFC Number Title Status
1 CSV Source Support for Delta Streamer COMPLETED
2 ORC Storage in Hudi COMPLETED
3 Timeline Service with Incremental File System View Syncing COMPLETED
4 Faster Hive incremental pull queries COMPLETED
5 HUI (Hudi WebUI) ABANDONED
6 Add indexing support to the log file ABANDONED
7 Point in time Time-Travel queries on Hudi table COMPLETED
8 Metadata based Record Index COMPLETED
9 Hudi Dataset Snapshot Exporter COMPLETED
10 Restructuring and auto-generation of docs COMPLETED
11 Refactor of the configuration framework of hudi project ABANDONED
12 Efficient Migration of Large Parquet Tables to Apache Hudi COMPLETED
13 Integrate Hudi with Flink COMPLETED
14 JDBC incremental puller COMPLETED
15 HUDI File Listing Improvements COMPLETED
16 Abstraction for HoodieInputFormat and RecordReader COMPLETED
17 Abstract common meta sync module support multiple meta service COMPLETED
18 Insert Overwrite API COMPLETED
19 Clustering data for freshness and query performance COMPLETED
20 handle failed records ONGOING
21 Allow HoodieRecordKey to be Virtual COMPLETED
22 Snapshot Isolation using Optimistic Concurrency Control for multi-writers COMPLETED
23 Hudi Observability metrics collection ABANDONED
24 Hoodie Flink Writer Proposal COMPLETED
25 Spark SQL Extension For Hudi COMPLETED
26 Optimization For Hudi Table Query COMPLETED
27 Data skipping index to improve query performance COMPLETED
28 Support Z-order curve COMPLETED
29 Hash Index COMPLETED
30 Batch operation ABANDONED
31 Hive integration Improvement ONGOING
32 Kafka Connect Sink for Hudi ONGOING
33 Hudi supports more comprehensive Schema Evolution COMPLETED
34 Hudi BigQuery Integration COMPLETED
35 Make Flink MOR table writing streaming friendly UNDER REVIEW
36 HUDI Metastore Server ONGOING
37 Hudi Metadata based Bloom Index ONGOING
38 Spark Datasource V2 Integration COMPLETED
39 Incremental source for Debezium COMPLETED
40 Hudi Connector for Trino COMPLETED
41 Hudi Snowflake Integration IN PROGRESS
42 Consistent Hashing Index ONGOING
43 Table Management Service IN PROGRESS
44 Hudi Connector for Presto COMPLETED
45 Asynchronous Metadata Indexing COMPLETED
46 Optimizing Record Payload Handling ONGOING
47 Add Call Produce Command for Spark SQL COMPLETED
48 LogCompaction for MOR tables ONGOING
49 Support sync with DataHub COMPLETED
50 Improve Timeline Server IN PROGRESS
51 Change Data Capture ONGOING
52 Introduce Secondary Index to Improve HUDI Query Performance ONGOING
53 Use Lock-Free Message Queue Improving Hoodie Writing Efficiency COMPLETED
54 New Table APIs and Streamline Hudi Configs UNDER REVIEW
55 Improve Hive/Meta sync class design and hierarchies COMPLETED
56 Early Conflict Detection For Multi-Writer COMPLETED
57 DeltaStreamer Protobuf Support COMPLETED
58 Integrate column stats index with all query engines UNDER REVIEW
59 Multiple event_time Fields Latest Verification in a Single Table UNDER REVIEW
60 Federated Storage Layer IN PROGRESS
61 Snapshot view management UNDER REVIEW
62 Diagnostic Reporter UNDER REVIEW
63 Functional Indexes UNDER REVIEW
64 New Hudi Table Spec API for Query Integrations UNDER REVIEW
65 Partition TTL Management UNDER REVIEW
66 Lockless Multi-Writer Support UNDER REVIEW
67 Hudi Bundle Standards UNDER REVIEW
68 A More Effective HoodieMergeHandler for COW Table with Parquet UNDER REVIEW
69 Hudi 1.x UNDER REVIEW
70 Hudi Reverse Streamer UNDER REVIEW
71 Enhance OCC conflict detection UNDER REVIEW
72 Redesign Hudi-Spark Integration ONGOING
73 Multi-Table Transactions UNDER REVIEW
74 HoodieStorage: Hudi Storage Abstraction and APIs UNDER REVIEW
75 Hudi-Native HFile Reader and Writer UNDER REVIEW
76 Auto Record key generation IN PROGRESS
77 Secondary Index UNDER REVIEW
78 Bridge release for 1.x IN PROGRESS
79 Robust handling of spark task retries and failures IN PROGRESS
80 Column Families UNDER REVIEW
81 Log Compaction with Merge Sort UNDER REVIEW
82 Concurrent schema evolution detection UNDER REVIEW