Create sub directories for ThirdPartyAudit dependency metadata

[finnegancarroll]

Description

The thirdPartyAudit task writes all dependency metadata to the same base directory of build/precommit/thirdPartyAudit/thirdPartyAudit/META-INF. When two separate dependencies attempt to write a file and directory at the same location the task fails with an error.

See #16840.

This PR handles collisions by appending a counter to duplicates.
i.e. dup-1.txt, dup-2.txt, ...

Related Issues

Resolves #16840

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 6f4639f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 4e866be: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 4b1b382: SUCCESS

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 10 lines in your changes missing coverage. Please review.

Project coverage is 72.29%. Comparing base (7050ecf) to head (060b6a2).
Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
...ensearch/gradle/precommit/ThirdPartyAuditTask.java	0.00%	10 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #16844      +/-   ##
============================================
+ Coverage     72.19%   72.29%   +0.09%     
- Complexity    65208    65263      +55     
============================================
  Files          5297     5297              
  Lines        303324   303328       +4     
  Branches      43913    43913              
============================================
+ Hits         218999   219292     +293     
+ Misses        66367    66106     -261     
+ Partials      17958    17930      -28     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[reta]

When two separate dependencies attempt to write a file and directory at the same location the task fails with an error.

The task should be run per module, each having own build directory, with no overlaps (I think). Do some modules share the build folder?

[finnegancarroll]

The task should be run per module, each having own build directory, with no overlaps (I think). Do some modules share the build folder?

It's not that plugins/modules are sharing build folders, but that the third party dependencies of an individual plugin/module are unzipped into a shared base folder during the third party audit task. Specifically plugins/<myplugin>/build/precommit/thirdPartyAudit/thirdPartyAudit.

In the minimal example linked in the issue these two randomly chosen dependencies happen to have jars which unzip a META-INF/LICENSE directory and META-INF/LICENSE file respectively.

dependencies {
  implementation "io.grpc:grpc-netty-shaded:${versions.grpc}"
  implementation "com.google.guava:guava:${versions.guava}"
}

[reta]

In the minimal example linked in the issue these two randomly chosen dependencies happen to have jars which unzip a META-INF/LICENSE directory and META-INF/LICENSE file respectively.

Oh I see, thank you, guava as a dependency is causing troubles everywhere

[github-actions]

❌ Gradle check result for 060b6a2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[finnegancarroll]

org.opensearch.index.engine.NRTReplicationEngineTests.testGetSegmentInfosSnapshotPreservesFilesUntilRelease is flaky

[github-actions]

✅ Gradle check result for 060b6a2: SUCCESS