Skip to content

Conversation

@NihalJain
Copy link
Contributor

@NihalJain NihalJain commented Oct 29, 2025

Summary of changes for this PoC (Up to date as of 30th Oct 2025 with 4 commits)

  • Commit 1: Make initial commit by copying hadoop trunk’s module hadoop-common-project/hadoop-auth into a new module hbase-auth-filters.

    • NOTE: This commit has no manual changes and is a true copy of commit #f3cd00d.
    • Purpose is to establish a local, reviewable baseline identical to Hadoop’s auth module for HBase-specific adjustments in next few commits.
  • Commit 2: Prune unnecessary pieces and wire the new module to HBase.

    • Drop unnecessary code/resources from copied code.
    • Adjust packages/imports to align with HBase usage.
    • Add dependencies to hbase-rest and hbase-http for hbase-auth-filters.
    • Temporarily drop tests in hbase-auth-filters to unblock initial review and avoid spending effort, if this PoC is declined.
    • Build note: Verified local build with mvn test -pl hbase-rest,hbase-http,hbase-auth-filters.
  • Commit 3: Address Error Prone to fix compilation issue.

  • Commit 4: Proactive Jakarta readiness to handle potential Hadoop migration; add reference script.

    • Copy more files from hadoop, needed to isolate javax.servlet vs. jakarta.servlet differences, keeping only required methods.
    • Files identified based on our organisations' (private) Hadoop with Jetty 12 / EE 10 efforts' code change.
    • Add check_hadoop_jakarta_impact.sh for reviewers' context (to be removed in follow-up commit).
    • Build note: Verified local build with mvn test -Dtest=TestFsDelegationToken -pl hbase-server; mvn test -pl hbase-http,hbase-auth-filters,hbase-rest,hbase-thrift.

Next steps (dependent on reviewer acceptance of this huge copy of hadoop-auth code for our decoupling initiative)

If PoC is acceptable:

  • Validate in real deployments with and without LDAP for the UI.
  • Decide on Curator usage strategy (defer or encapsulate); track with a follow-up task?
  • Clean up the new module’s POM (dependencies, plugins).
  • Restore and adapt relevant tests to the HBase test framework.
  • Fix imports and formatting to match our conventions.
  • We could potentially refactor/move all servlet filter related classes lying around the code base into this new module as follow up task!

If PoC is not acceptable:

  • Consider shading to avoid bulk copying. Refer HBASE-29563

CC: @Apache9, @stoty, @apurtell, @ndimiduk

- Make initial commit with hadoop trunk for module hadoop-common-project/hadoop-auth as of commit f3cd00d8db6351dabed286d07f9285a31fc1ecb7as is!
…pendency to new module as needed, drop tests from hbase-auth-filters temporarily (plan is to get first round of review before adjusting/fixing)
@NihalJain
Copy link
Contributor Author

Build code locally and ran following tests:

mvn test -pl hbase-rest,hbase-http,hbase-auth-filters

Waiting for build result to see if all works fine.

Also, if the amount of new code being copied looks acceptable, can work on following efforts to get this in:

  • Test on a real setup with/without ldap for ui
  • Figure out how we deal with Curator usage. Relax for now, do as another task?
  • Clean pom.xml for the new module
  • Add back tests/classes, whichever are relevant and fix them as per hbase test framework
  • Fix imports
  • Reformat code
  • etc

@NihalJain NihalJain marked this pull request as draft October 29, 2025 14:40
@NihalJain NihalJain changed the title WIP: HBASE-29557 Decouple dependency on Hadoop AuthenticationFilter classes HBASE-29557 Decouple dependency on Hadoop AuthenticationFilter classes Oct 29, 2025
@Apache-HBase

This comment was marked as outdated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 14s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+0 🆗 mvndep 0m 33s Maven dependency ordering for branch
+1 💚 mvninstall 5m 54s master passed
+1 💚 compile 18m 50s master passed
+1 💚 checkstyle 2m 47s master passed
+1 💚 spotbugs 13m 49s master passed
+1 💚 spotless 1m 6s branch has no errors when running spotless:check.
-0 ⚠️ patch 1m 29s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 16s Maven dependency ordering for patch
+1 💚 mvninstall 4m 43s the patch passed
+1 💚 compile 14m 2s the patch passed
-0 ⚠️ javac 14m 2s /results-compile-javac-root.txt root generated 73 new + 1896 unchanged - 0 fixed = 1969 total (was 1896)
-0 ⚠️ blanks 0m 0s /blanks-eol.txt The patch has 7 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️ checkstyle 2m 47s /results-checkstyle-root.txt root: The patch generated 188 new + 1 unchanged - 0 fixed = 189 total (was 1)
+1 💚 xmllint 0m 1s No new issues.
+1 💚 spotbugs 15m 11s the patch passed
+1 💚 hadoopcheck 14m 50s Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
-1 ❌ spotless 0m 10s patch has 33 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 💚 asflicense 0m 43s The patch does not generate ASF License warnings.
105m 33s
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7422/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #7422
Optional Tests dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile spotbugs checkstyle hbaseanti
uname Linux dfcd4d802870 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e90af6b
Default Java Eclipse Adoptium-17.0.11+9
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7422/2/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 163 (vs. ulimit of 30000)
modules C: hbase-auth-filters . hbase-http hbase-rest U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7422/2/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@NihalJain
Copy link
Contributor Author

Added a new commit handling few more cases based on following analysis as per an internal change having hadoop with jetty12 with EE10:
image
Also ran following tests to verify no regression in UTs:

mvn test -Dtest=TestFsDelegationToken -pl hbase-server -o
mvn test -pl hbase-http,hbase-auth-filters,hbase-rest,hbase-thrift -o

@NihalJain
Copy link
Contributor Author

NihalJain commented Oct 30, 2025

Could you please review this PR and share your feedback? Please read the commit description for full context. In particular, do we agree that the code copy is acceptable?

… reference script.

  - Copy more files from hadoop, needed to isolate javax.servlet vs. jakarta.servlet differences, keeping only required methods.
  - Files identified based on our organisations' (private) Hadoop with Jetty 12 / EE 10 efforts' code change.
  - Add `check_hadoop_jakarta_impact.sh` for reviewers' context (to be removed in follow-up commit).
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 17s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+0 🆗 mvndep 0m 26s Maven dependency ordering for branch
+1 💚 mvninstall 8m 28s master passed
+1 💚 compile 25m 25s master passed
+1 💚 checkstyle 4m 46s master passed
+1 💚 spotbugs 24m 3s master passed
+1 💚 spotless 1m 49s branch has no errors when running spotless:check.
-0 ⚠️ patch 2m 37s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 35s Maven dependency ordering for patch
+1 💚 mvninstall 7m 25s the patch passed
+1 💚 compile 27m 6s the patch passed
-0 ⚠️ javac 27m 6s /results-compile-javac-root.txt root generated 73 new + 1896 unchanged - 0 fixed = 1969 total (was 1896)
-0 ⚠️ blanks 0m 0s /blanks-eol.txt The patch has 9 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️ checkstyle 4m 55s /results-checkstyle-root.txt root: The patch generated 202 new + 1 unchanged - 0 fixed = 203 total (was 1)
-0 ⚠️ shellcheck 0m 1s /results-shellcheck.txt The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚 xmllint 0m 1s No new issues.
+1 💚 spotbugs 25m 47s the patch passed
+1 💚 hadoopcheck 19m 44s Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
-1 ❌ spotless 0m 13s patch has 33 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
-1 ❌ asflicense 1m 5s /results-asflicense.txt The patch generated 1 ASF License warnings.
165m 47s
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7422/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #7422
Optional Tests dupname asflicense codespell detsecrets shellcheck shelldocs spotless javac xmllint hadoopcheck compile spotbugs checkstyle hbaseanti
uname Linux d08061d94c6f 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / c0bb774
Default Java Eclipse Adoptium-17.0.11+9
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7422/3/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 162 (vs. ulimit of 30000)
modules C: hbase-auth-filters hbase-http hbase-mapreduce hbase-rest . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7422/3/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 shellcheck=0.8.0 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 15s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+0 🆗 mvndep 0m 26s Maven dependency ordering for branch
+1 💚 mvninstall 5m 58s master passed
+1 💚 compile 17m 1s master passed
+1 💚 checkstyle 2m 51s master passed
+1 💚 spotbugs 14m 56s master passed
+1 💚 spotless 1m 10s branch has no errors when running spotless:check.
-0 ⚠️ patch 1m 41s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 19s Maven dependency ordering for patch
+1 💚 mvninstall 4m 52s the patch passed
+1 💚 compile 14m 28s the patch passed
-0 ⚠️ javac 14m 28s /results-compile-javac-root.txt root generated 73 new + 1896 unchanged - 0 fixed = 1969 total (was 1896)
-0 ⚠️ blanks 0m 0s /blanks-eol.txt The patch has 9 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️ checkstyle 2m 43s /results-checkstyle-root.txt root: The patch generated 202 new + 1 unchanged - 0 fixed = 203 total (was 1)
-0 ⚠️ shellcheck 0m 1s /results-shellcheck.txt The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚 xmllint 0m 1s No new issues.
+1 💚 spotbugs 16m 14s the patch passed
+1 💚 hadoopcheck 14m 33s Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
-1 ❌ spotless 0m 10s patch has 33 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
-1 ❌ asflicense 0m 55s /results-asflicense.txt The patch generated 1 ASF License warnings.
106m 58s
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7422/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #7422
Optional Tests dupname asflicense codespell detsecrets shellcheck shelldocs spotless javac xmllint hadoopcheck compile spotbugs checkstyle hbaseanti
uname Linux b4cbf7fb45e7 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 0a2cbfa
Default Java Eclipse Adoptium-17.0.11+9
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7422/4/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 162 (vs. ulimit of 30000)
modules C: hbase-auth-filters hbase-http hbase-mapreduce hbase-rest . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7422/4/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3 shellcheck=0.8.0 xmllint=20913
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@stoty
Copy link
Contributor

stoty commented Nov 3, 2025

I think that the (mostly) copied new files should have a boilerplate comment explaining where they were copied from.

@stoty
Copy link
Contributor

stoty commented Nov 3, 2025

Otherwise it looks fine to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants