Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: [#53] Create parallel gradle build for CI #149

Merged
merged 77 commits into from
Nov 15, 2024

Conversation

ursjoss
Copy link
Collaborator

@ursjoss ursjoss commented Oct 6, 2024

Resolves #53

Summary

Keeping the regular ant-based build as is, this PR adds a parallel gradle configuration of the project with the purpose of allowing the CI to compile the project and run the unit tests in considerably less time (<4 minutes intstead of 20 minutes for the CI). Also, it runs the build and the tests both with Java 17 and Java 21.

To run the build locally:

To run all tests

With just installed:

just verify

Without just installed:

./gradlew check -Dtest.profile=allTests

To run headless tests (as on CI)

With just installed:

just verify-ci

Without just installed:

./gradlew check

Details and Context

TR is an ant-based Netbeans Platform project. Our current way of using CI is very cumbersome and slow (the build takes around 20 minutes). The root cause is most likely my very limited experience with ant-based projects and therefore an inefficient usage of leveraging the possibilities provided by ant. While the current approach helped us getting a basic CI that also fails the build with failing tests and therefore was first step in the right direction, I'd like to improve build times using gradle.

I experimented with migrating fully from ant to gradle (see ongoing branch https://github.com/ursjoss/tr-pc/tree/spike/gradle.). I would see this as a mid-term goal, but currently, there are for too many unknowns that will need to be tackled over time. In the mean time, this PR introduces a parallel build using gradle. We're not touching the ant build system configuration for now and still rely on it to run the application, create installers etc.

The current gradle configuration covers dependencies the gradle way up to the point where we can compile each TR module and run the tests. It does not yet cover running the application. Also, the third-party-dependencies sitting in the project as a module are ignored in the gradle build. I tried to declare the same versions of dependencies that we find in the netbeans module declarations, but that wasn't always possible.

Let's e.g. take fop. The netbeans module uses version 1.1. You can see the dependencies in modules/fop/release/modules/ext/. fop-1.1 depends on avalon-framework-4.2.0.jar. fop-1.1 on maven central, however, depends on org.apache.avalon.framework:avalon-framework-api and org.apache.avalon.framework:avalon-framework-impl in version 4.2.0. And those are not available on maven central anymore (only version 4.3.1).

We'll need to update dependencies both as netbeans modules and for the gradle build to be in sync to be on the safe side. See #151 for FOP.

With this, CI takes below 4 minutes instead of ~20.

Note regarding four commented out tests:

  • modules/au.com.trgtd.tr.l10n.en_US/test/unit/src/au/com/trgtd/tr/l10n/en_US/OverviewSVGProvider_en_USTest.java
  • modules/au.com.trgtd.tr.l10n.es_ES/test/unit/src/au/com/trgtd/tr/l10n/es_ES/OverviewSVGProvider_es_ESTest.java
  • modules/au.com.trgtd.tr.l10n.fr_FR/test/unit/src/au/com/trgtd/tr/l10n/fr_FR/OverviewSVGProvider_fr_FRTest.java

It seems that those tests were not executed in the ant build. I added a TODO to fix the testGetURL() tests in those three classes.

  • modules/au.com.trgtd.tr.util/test/unit/src/au/com/trgtd/tr/util/ObservableImplTest.java

canConcurrentlyModifyObserversWhileNotifyingAndStillNotifiesAll() started failing in this branch - but only if run on JDK 21 (not 17). If I switch the Map implementation in ObservableImpl for observers from WeakHashMap to HashMap, it works as expected. Not 100% sure if this test is an actual specification though. I commented the test out for now.

Note that in this PR, I purposely added a commit to prove that a failing test breaks the build (see here) (and reverted it after that).

@ursjoss ursjoss changed the title chore: Create parallel gradle build for CI chore: [#150] Create parallel gradle build for CI Oct 6, 2024
@ursjoss ursjoss requested a review from RadekCap October 6, 2024 12:58
@ursjoss ursjoss changed the title chore: [#150] Create parallel gradle build for CI chore: [#53] Create parallel gradle build for CI Oct 6, 2024
@RadekCap
Copy link
Collaborator

Good to go.

`just verify
416 actionable tasks: 416 executed

just verify-ci
416 actionable tasks: 2 executed, 414 up-to-date
`

The application is compileable and runnable with just. I haven't figured out how to run it from Netbeans as it is loaded as a Gradle project. It's probably because of my knowledge of Netbeans.

build.gradle.kts Show resolved Hide resolved
@RadekCap RadekCap merged commit 857866f into thinkingrock-gtd:main Nov 15, 2024
2 checks passed
@ursjoss ursjoss deleted the feat/gradle4ci branch November 15, 2024 16:24
@ursjoss
Copy link
Collaborator Author

ursjoss commented Nov 15, 2024

@RadekCap Many thanks for your review and testing. Much appreciated.

@ursjoss
Copy link
Collaborator Author

ursjoss commented Nov 18, 2024

The application is compileable and runnable with just. I haven't figured out how to run it from Netbeans as it is loaded as a Gradle project. It's probably because of my knowledge of Netbeans.

@RadekCap I guess you're referring to what is now described in #153. I have no clue how I did not run into this when working on this PR, but somehow I didn't until now. Can you confirm?

@RadekCap
Copy link
Collaborator

I have the same experience. It's new since this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Build: Improve the duration of the continuous integration workflows
2 participants