Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Toolchain downloads frequently failing #7492

Open
cpovirk opened this issue Oct 26, 2024 · 2 comments
Open

Toolchain downloads frequently failing #7492

cpovirk opened this issue Oct 26, 2024 · 2 comments
Labels
P3 package=general type=defect Bug, not working as expected

Comments

@cpovirk
Copy link
Member

cpovirk commented Oct 26, 2024

I've had multiple PRs recently that have failed testing because of trouble downloading a JDK. Here's the most recent example, which downloaded 11 but failed on 23, but note that I think I've seen failures for 11 and have definitely seen a failure for 8:

[INFO] --- toolchains:4.5.0:toolchain (download-11) @ guava ---
[INFO] Required toolchain: jdk [ vendor='zulu' version='11' ]
[INFO] Begin to install JDK 11
[INFO] Download zulu11.76.21-ca-jdk11.0.25-linux_x64.tar.gz from https://cdn.azul.com/zulu/bin/zulu11.76.21-ca-jdk11.0.25-linux_x64.tar.gz
[INFO] Extract zulu11.76.21-ca-jdk11.0.25-linux_x64.tar.gz
[INFO] JDK installed: /home/runner/.m2/jdks/zulu11.76.21-ca-jdk11.0.25-linux_x64
[INFO] 
[INFO] --- toolchains:4.5.0:toolchain (download-23-and-surefire-version) @ guava ---
[INFO] Required toolchain: jdk [ vendor='zulu' version='23' ]
[INFO] Begin to install JDK 23
[INFO] Download zulu23.30.13-ca-jdk23.0.1-linux_x64.tar.gz from https://cdn.azul.com/zulu/bin/zulu23.30.13-ca-jdk23.0.1-linux_x64.tar.gz
[INFO] Extract zulu23.30.13-ca-jdk23.0.1-linux_x64.tar.gz
Error:  Failed to download and install JDK
org.codehaus.plexus.archiver.ArchiverException: Error while expanding /home/runner/.m2/jdks/zulu23.30.13-ca-jdk23.0.1-linux_x64.tar.gz
    at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute (TarUnArchiver.java:124)
    at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute (TarUnArchiver.java:88)
    at org.codehaus.plexus.archiver.AbstractUnArchiver.extract (AbstractUnArchiver.java:151)
    at org.apache.maven.plugins.toolchain.FoojayService.extractArchiveFile (FoojayService.java:206)
    at org.apache.maven.plugins.toolchain.FoojayService.downloadAndExtract (FoojayService.java:166)
    at org.apache.maven.plugins.toolchain.FoojayService.downloadAndExtractJdk (FoojayService.java:94)
    at org.apache.maven.plugins.toolchain.ToolchainMojo.autoInstallJdk (ToolchainMojo.java:233)
    at org.apache.maven.plugins.toolchain.ToolchainMojo.selectToolchain (ToolchainMojo.java:195)
    at org.apache.maven.plugins.toolchain.ToolchainMojo.execute (ToolchainMojo.java:111)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:255)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:201)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:361)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:314)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:52)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:161)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:73)
Caused by: java.io.EOFException: Unexpected end of ZLIB input stream
    at java.util.zip.InflaterInputStream.fill (InflaterInputStream.java:266)
    at java.util.zip.InflaterInputStream.read (InflaterInputStream.java:175)
    at java.util.zip.GZIPInputStream.read (GZIPInputStream.java:128)
    at java.io.BufferedInputStream.fill (BufferedInputStream.java:291)
    at java.io.BufferedInputStream.read1 (BufferedInputStream.java:347)
    at java.io.BufferedInputStream.implRead (BufferedInputStream.java:420)
    at java.io.BufferedInputStream.read (BufferedInputStream.java:399)
    at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read (TarArchiveInputStream.java:873)
    at java.io.InputStream.read (InputStream.java:220)
    at org.codehaus.plexus.util.IOUtil.copy (IOUtil.java:189)
    at org.codehaus.plexus.util.IOUtil.copy (IOUtil.java:175)
    at org.codehaus.plexus.archiver.AbstractUnArchiver.extractFile (AbstractUnArchiver.java:372)
    at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute (TarUnArchiver.java:114)
    at org.codehaus.plexus.archiver.tar.TarUnArchiver.execute (TarUnArchiver.java:88)
    at org.codehaus.plexus.archiver.AbstractUnArchiver.extract (AbstractUnArchiver.java:151)
    at org.apache.maven.plugins.toolchain.FoojayService.extractArchiveFile (FoojayService.java:206)
    at org.apache.maven.plugins.toolchain.FoojayService.downloadAndExtract (FoojayService.java:166)
    at org.apache.maven.plugins.toolchain.FoojayService.downloadAndExtractJdk (FoojayService.java:94)
    at org.apache.maven.plugins.toolchain.ToolchainMojo.autoInstallJdk (ToolchainMojo.java:233)
    at org.apache.maven.plugins.toolchain.ToolchainMojo.selectToolchain (ToolchainMojo.java:195)
    at org.apache.maven.plugins.toolchain.ToolchainMojo.execute (ToolchainMojo.java:111)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:255)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:201)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:361)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:314)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:52)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:161)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:73)
Error:  No toolchain matched from 2 found for toolchainType jdk
[INFO] Required toolchain: jdk [ vendor='zulu' version='21' ]
[INFO] Found matching toolchain for toolchainType jdk: JDK[/opt/hostedtoolcache/Java_Zulu_jdk/21.0.5-11/x64]
Error:  Cannot find matching toolchain definitions for the following toolchain types:
jdk [ vendor='zulu' version='23' ]

(Aside: GitHub is inserting Markdown when I copy and paste directly from the Actions results, possibly to link to anchors in that page? Not that that would work inside the code block that I'm pasting into. I ended up pasting into a plain-text window first and then copying and pasting again.)

The automatic toolchain downloads are nice, but it would probably be more reliable for CI to download all the toolchains we need with setup-java. (That would likely also be faster, as noted in internal bug 334979149.) The trick would be ensuring that we still use the toolchain that we want for (a) testing and (b, possibly less important) running Maven itself. Error Prone has this worked out by being careful about the order in which they list their downloaded toolchains and then (I'm guessing) by configuring maven-surefire-plugin more directly than we do.

(We could still keep the toolchain plugin for local builds. It's much nicer for users than if they have to download various toolchains themselves.)
@cpovirk cpovirk added P3 package=general type=defect Bug, not working as expected labels Oct 26, 2024
@ben-manes
Copy link
Contributor

Also consider trying to ensure that you are leveraging the hosted tool cache which avoids downloading if you use one of the pre-installed jdks (temurin 8, 11, 17, 21). You can also use composite github actions for reuse, optionally across repositories, if you find a common setup is helpful (e.g. I wrangle all of my jdk versions in this action)

@cpovirk
Copy link
Member Author

cpovirk commented Dec 12, 2024

Slightly different stack trace today:

java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
    at jdk.internal.util.Preconditions.outOfBounds (Preconditions.java:64)
    at jdk.internal.util.Preconditions.outOfBoundsCheckIndex (Preconditions.java:70)
    at jdk.internal.util.Preconditions.checkIndex (Preconditions.java:266)
    at java.util.Objects.checkIndex (Objects.java:361)
    at java.util.ArrayList.get (ArrayList.java:427)
    at com.google.gson.JsonArray.get (JsonArray.java:203)
    at org.apache.maven.plugins.toolchain.FoojayService.parseFileNameAndDownloadUrl (FoojayService.java:145)
    at org.apache.maven.plugins.toolchain.FoojayService.downloadAndExtractJdk (FoojayService.java:82)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
P3 package=general type=defect Bug, not working as expected
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ben-manes @cpovirk and others