HIVE-29274: Flaky TestMetastoreLeaseLeader#testHouseKeepingThreads #6142

dengzhhu653 · 2025-10-18T12:03:32Z

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

No

How was this patch tested?

https://ci.hive.apache.org/job/hive-flaky-check/915

...unit/src/test/java/org/apache/hadoop/hive/metastore/MetastoreHousekeepingLeaderTestBase.java

...ive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetastoreHousekeepingLeader.java

...tore-server/src/main/java/org/apache/hadoop/hive/metastore/leader/LeaderElectionContext.java

...astore-server/src/main/java/org/apache/hadoop/hive/metastore/leader/LeaseLeaderElection.java

deniskuzZ

added comments

...astore-server/src/main/java/org/apache/hadoop/hive/metastore/leader/LeaseLeaderElection.java

...astore-server/src/main/java/org/apache/hadoop/hive/metastore/PersistenceManagerProvider.java

sonarqubecloud · 2025-10-24T07:52:52Z

Quality Gate passed

Issues
22 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

deniskuzZ · 2025-10-25T15:47:55Z

...tore-server/src/main/java/org/apache/hadoop/hive/metastore/leader/LeaderElectionContext.java

  public enum TTYPE {
    HOUSEKEEPING(new TableName(Warehouse.DEFAULT_CATALOG_NAME, "__METASTORE_LEADER_ELECTION__",
        "metastore_housekeeping"), "housekeeping"),
-    WORKER(new TableName(Warehouse.DEFAULT_CATALOG_NAME, "__METASTORE_LEADER_ELECTION__",


nice! we forgot to drop this

deniskuzZ · 2025-10-25T15:50:16Z

...tore-server/src/main/java/org/apache/hadoop/hive/metastore/leader/LeaderElectionContext.java

-        listenerMap.entrySet()) {
-      List<LeadershipStateListener> listenerList = entry.getValue();
+    List<TTYPE> ttypes = new ArrayList<>(listeners.keySet());
+    Collections.shuffle(ttypes);


why do we need to shuffle?

deniskuzZ · 2025-10-25T15:51:51Z

...unit/src/test/java/org/apache/hadoop/hive/metastore/MetastoreHousekeepingLeaderTestBase.java

-    MetastoreConf.setVar(conf, ConfVars.METASTORE_HOUSEKEEPING_LEADER_HOSTNAME, leaderHostName);
    MetastoreConf.setVar(conf, ConfVars.METASTORE_HOUSEKEEPING_LEADER_ELECTION,
-        configuredLeader ? "host" : "lock");
+        leaderHostName != null ? "host" : "lock");


please try use enum (i.e Method from LeaderElectionFactory)

deniskuzZ · 2025-10-25T15:59:50Z

...tastore-server/src/test/java/org/apache/hadoop/hive/metastore/leader/TestLeaderElection.java

-    wait(service, flag1, flag2);
-    // now instance1 lease is timeout, the instance2 should be leader now
+    instance1.close();
+    synchronized (flag2) {


why do you need synchronized here? AtomicBoolean is designed for lock-free atomic updates.

while (!flag2.get()) { Thread.onSpinWait(); // JDK 9+ }

deniskuzZ · 2025-10-25T16:03:04Z

...astore-server/src/main/java/org/apache/hadoop/hive/metastore/leader/LeaseLeaderElection.java

  private static final Logger LOG = LoggerFactory.getLogger(LeaseLeaderElection.class);

-  private static final AtomicLong ID = new AtomicLong();
+  protected static final AtomicLong ID = new AtomicLong();


won't it be better if we expose protected fields via getters?

deniskuzZ

LGTM +1, some minor suggestions

asf-ci-hive added tests pending tests passed and removed tests pending labels Oct 18, 2025