Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] Create sub directories for ThirdPartyAudit dependency metadata #16911

Merged
merged 1 commit into from
Dec 24, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -229,8 +229,7 @@
@TaskAction
public void runThirdPartyAudit() throws IOException {
Set<File> jars = getJarsToScan();

extractJars(jars);
Set<File> extractedJars = extractJars(jars);

Check warning on line 232 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L232

Added line #L232 was not covered by tests

final String forbiddenApisOutput = runForbiddenAPIsCli();

Expand All @@ -248,7 +247,7 @@

Set<String> jdkJarHellClasses = null;
if (this.jarHellEnabled) {
jdkJarHellClasses = runJdkJarHellCheck();
jdkJarHellClasses = runJdkJarHellCheck(extractedJars);

Check warning on line 250 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L250

Added line #L250 was not covered by tests
}

if (missingClassExcludes != null) {
Expand Down Expand Up @@ -302,16 +301,26 @@
getLogger().error("Forbidden APIs output:\n{}==end of forbidden APIs==", forbiddenApisOutput);
}

private void extractJars(Set<File> jars) {
/**
* Extract project jars to build directory as specified by getJarExpandDir.
* Handle multi release jars by keeping versions closest to `targetCompatibility` version.
* @param jars to extract to build dir
* @return File set of extracted jars
*/
private Set<File> extractJars(Set<File> jars) {
Set<File> extractedJars = new TreeSet<>();

Check warning on line 311 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L311

Added line #L311 was not covered by tests
File jarExpandDir = getJarExpandDir();
// We need to clean up to make sure old dependencies don't linger
getProject().delete(jarExpandDir);

jars.forEach(jar -> {
String jarPrefix = jar.getName().replace(".jar", "");
File jarSubDir = new File(jarExpandDir, jarPrefix);
extractedJars.add(jarSubDir);

Check warning on line 319 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L317-L319

Added lines #L317 - L319 were not covered by tests
FileTree jarFiles = getProject().zipTree(jar);
getProject().copy(spec -> {
spec.from(jarFiles);
spec.into(jarExpandDir);
spec.into(jarSubDir);

Check warning on line 323 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L323

Added line #L323 was not covered by tests
// exclude classes from multi release jars
spec.exclude("META-INF/versions/**");
});
Expand All @@ -330,14 +339,16 @@
Integer.parseInt(targetCompatibility.get().getMajorVersion())
).forEach(majorVersion -> getProject().copy(spec -> {
spec.from(getProject().zipTree(jar));
spec.into(jarExpandDir);
spec.into(jarSubDir);

Check warning on line 342 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L342

Added line #L342 was not covered by tests
String metaInfPrefix = "META-INF/versions/" + majorVersion;
spec.include(metaInfPrefix + "/**");
// Drop the version specific prefix
spec.eachFile(details -> details.setPath(details.getPath().replace(metaInfPrefix, "")));
spec.setIncludeEmptyDirs(false);
}));
});

return extractedJars;

Check warning on line 351 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L351

Added line #L351 was not covered by tests
}

private void assertNoJarHell(Set<String> jdkJarHellClasses) {
Expand Down Expand Up @@ -399,7 +410,12 @@
return forbiddenApisOutput;
}

private Set<String> runJdkJarHellCheck() throws IOException {
/**
* Execute java with JDK_JAR_HELL_MAIN_CLASS against provided jars with OpenSearch core in the classpath.
* @param jars to scan for jarHell violations.
* @return standard out of jarHell process.
*/
private Set<String> runJdkJarHellCheck(Set<File> jars) throws IOException {
ByteArrayOutputStream standardOut = new ByteArrayOutputStream();
InjectedExecOps execOps = getProject().getObjects().newInstance(InjectedExecOps.class);
ExecResult execResult = execOps.getExecOps().javaexec(spec -> {
Expand All @@ -408,9 +424,8 @@
getRuntimeConfiguration(),
getProject().getConfigurations().getByName(CompileOnlyResolvePlugin.RESOLVEABLE_COMPILE_ONLY_CONFIGURATION_NAME)
);

spec.getMainClass().set(JDK_JAR_HELL_MAIN_CLASS);
spec.args(getJarExpandDir());
spec.args(jars);

Check warning on line 428 in buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java

View check run for this annotation

Codecov / codecov/patch

buildSrc/src/main/java/org/opensearch/gradle/precommit/ThirdPartyAuditTask.java#L428

Added line #L428 was not covered by tests
spec.setIgnoreExitValue(true);
if (javaHome != null) {
spec.setExecutable(javaHome + "/bin/java");
Expand Down
Loading