Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CELEBORN-898][INFRA] Fix java.lang.NoClassDefFoundError: org/hamcrest/SelfDescribing for SBT CI #1818

Closed
wants to merge 3 commits into from

Conversation

cfmcgrady
Copy link
Contributor

What changes were proposed in this pull request?

As title

Why are the changes needed?

Recently, I came across an issue in the SBT CI process that can result in failure due to the NoClassDefFoundError exception.

[error] Uncaught exception when running org.apache.celeborn.common.unsafe.PlatformUtilSuite: java.lang.NoClassDefFoundError: org/hamcrest/SelfDescribing
[error] sbt.ForkMain$ForkError: java.lang.NoClassDefFoundError: org/hamcrest/SelfDescribing
[error]    at java.lang.ClassLoader.defineClass1(Native Method)
[error]    at java.lang.ClassLoader.defineClass(ClassLoader.java:757)
[error]    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
[error]    at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
[error]    at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
[error]    at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
[error]    at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
[error]    at java.security.AccessController.doPrivileged(Native Method)
[error]    at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
[error]    at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
[error]    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
[error]    at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
[error]    at org.junit.runner.Computer.getSuite(Computer.java:28)
[error]    at org.junit.runner.Request.classes(Request.java:77)
[error]    at org.junit.runner.Request.classes(Request.java:92)
[error]    at com.novocode.junit.JUnitTask.execute(JUnitTask.java:52)
[error]    at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
[error]    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error]    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error]    at java.lang.Thread.run(Thread.java:750)
[error] Caused by: sbt.ForkMain$ForkError: java.lang.ClassNotFoundException: org.hamcrest.SelfDescribing
[error]    at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
[error]    at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
[error]    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
[error]    at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
[error]    at java.lang.ClassLoader.defineClass1(Native Method)
[error]    at java.lang.ClassLoader.defineClass(ClassLoader.java:757)
[error]    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
[error]    at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
[error]    at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
[error]    at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
[error]    at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
[error]    at java.security.AccessController.doPrivileged(Native Method)
[error]    at java.net.URLClassLoader.findClass(URLClassLoader.java:362)

Upon further investigation, I found that the root cause is SBT's sometimes inability to resolve Maven dependencies cached within GA.

./build/sbt "show celeborn-common/update"
[info] 		org.hamcrest:hamcrest-core:1.3:default: (MISSING) Artifact(hamcrest-core, jar, jar, None, Vector(), Some(file:/home/runner/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar), Map(), None, false)

This PR addresses the random issue by disabling the Maven cache for SBT CI.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass GA

#1797 pass GA after disabled maven cache.

@cfmcgrady cfmcgrady changed the title [CELEBORN-XXX][INFRA]fix java.lang.NoClassDefFoundError: org/hamcrest/SelfDescribing for SBT CI [CELEBORN-XXX][INFRA] Fix java.lang.NoClassDefFoundError: org/hamcrest/SelfDescribing for SBT CI Aug 15, 2023
@cfmcgrady cfmcgrady changed the title [CELEBORN-XXX][INFRA] Fix java.lang.NoClassDefFoundError: org/hamcrest/SelfDescribing for SBT CI [CELEBORN-898][INFRA] Fix java.lang.NoClassDefFoundError: org/hamcrest/SelfDescribing for SBT CI Aug 15, 2023
@codecov
Copy link

codecov bot commented Aug 15, 2023

Codecov Report

Merging #1818 (eb8fe2b) into main (57fdbf0) will not change coverage.
Report is 1 commits behind head on main.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1818   +/-   ##
=======================================
  Coverage   46.45%   46.45%           
=======================================
  Files         163      163           
  Lines       10135    10135           
  Branches      934      934           
=======================================
  Hits         4707     4707           
  Misses       5117     5117           
  Partials      311      311           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@pan3793
Copy link
Member

pan3793 commented Aug 15, 2023

Sounds like a dirty cache issue, are you plan to disable the cache permanently?

@cfmcgrady
Copy link
Contributor Author

Sounds like a dirty cache issue, are you plan to disable the cache permanently?

We can enable the cache for SBT CI in the separate PR using actions/cache instead of actions/setup-java.

@cfmcgrady
Copy link
Contributor Author

thanks, merging to branch main(v0.4.0)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants