HIVE-28668: Hive should emit fewer events for truncate table operation #5582
Conversation
asf-ci-hive
added
tests pending
tests passed
and removed
tests failed
tests pending
labels
| } | ||
| alterHandler.alterPartition(ms, wh, catName, dbName, tableName, null, partition, | ||
| environmentContext, this, validWriteIds); | ||
| alterHandler.alterPartitions(ms, wh, catName, dbName, tableName, partitions, environmentContext, |
There was a problem hiding this comment.
since we know this is a truncation op, can we just call ms.alterPartitions(catName, dbname, name, partValsList, new_parts, writeId, writeIdList) and create the event directly? same for truncating the non-partitioned table.
There was a problem hiding this comment.
We'll also call ObjectStore.alterPartitions() where the operation may not be a truncate, for e.g: L#6111 and L#6263
There was a problem hiding this comment.
this operation is in a context of truncation, and we have done almost every thing in HMSHandler, like reset the column stats, truncate the table directory, so in my view, we don't need to call alterHandler.alterPartitions to perform other actions, such as fetch the partitions from db again, update the stats, etc
There was a problem hiding this comment.
I have implemented your suggestions, Can you please verify the latest changes?
...astore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
Outdated
Show resolved
Hide resolved
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/TruncatePartitionHandler.java
Outdated
Show resolved
Hide resolved
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/TruncatePartitionHandler.java
Outdated
Show resolved
Hide resolved
asf-ci-hive
added
tests pending
tests unstable
and removed
tests passed
tests pending
labels
asf-ci-hive
added
tests pending
tests failed
and removed
tests unstable
tests pending
labels
asf-ci-hive
added
tests pending
tests passed
and removed
tests failed
tests pending
labels
|
What changes were proposed in this pull request?
Currently, we emit twice the number of events for each table/partition for truncate operation. This patch removes this redundant events.
Why are the changes needed?
Having fewer events help external systems that consume these events to catch up with the quantum of events faster.
Does this PR introduce any user-facing change?
No.
Is the change a dependency upgrade?
No.
How was this patch tested?
Added unit test for the patch