[Java][CI] Building C++ libraries on Ubuntu aarch64 job takes 3+ hours to complete in java-jars

[danepitkin]

Describe the enhancement requested

Can we speed this up? Other OS/architectures take 10-30min.

See java-jars job in crossbow: https://github.com/ursacomputing/crossbow/actions/runs/9817940199/job/27109855273

Component(s)

Continuous Integration

[assignUser]

It's the vcpkg builds. If you look at the time stamps arrow itself only takes 30 minutes. But llvm takes over an hour https://github.com/ursacomputing/crossbow/actions/runs/9817940199/job/27109855273#step:8:5926 and abseil half an hour etc.

I'd recommend adding vcpkg binary caching through the nuget backend like I did for other jobs: apache/arrow#13507

Docker complicates things a bit here, the easiest way is probably to mount the vcpkg directory so you don't have pass things into docker etc.?

[danepitkin]

Thank you! I'll take a look at this.

[pitrou]

You're building in release mode, and also building the C++ tests, which probably explains part of the build duration.

Also, in the JNI build you've linked to, the Arrow C++ build takes 25 minutes (from 12:05 to 12:30).

[danepitkin]

I noticed the java-jars job is back down to ~30min. I wonder if someone fixed this already? See example here: apache/arrow#44023

[pitrou]

@danepitkin It's just that in some cases the vcpkg step is successfully cached in the Docker image, and in some cases it's not.

There is an issue open to improve this: apache/arrow#43951

[pitrou]

@danepitkin Is it relevant to keep this in the main apache/arrow repo, or should it be moved to arrow-java?

[kou]

Let's move to apache/arrow-java.