Escalating background tasks "overriding microprofile classpath" causing high load and stalling builds

[E14]

For some time I'm seeing an issue where the "Overriding MicroProfile classpath...." ~ "Collecting Quarkus deployment dependencies" background task

• Often gets stuck at a specific dependency and does not continue
• Is run multiple times concurrently
• Causes high load

These may be related to having multiple IntelliJ projects open at the same time.

When this situation starts, an additional background process is started basically every time any source (and I think resource) file is saved.

Please let me know if any additional data is needed to resolve this, sadly I don't know how to reproduce the behaviour as it happens sporadically.

[angelozerr]

This problem is known and it is so bad!!!! See #1143

I will work soon on this Quarkus deploiement support to try to fix this awful performance problem and fix #1189

[E14]

@angelozerr oh, I really tried to find this issue :)

Thanks for the pointer, I'll watch that issue then!

[E14]

Lets close this as a duplicate of #1143 then, let me know if you need logs (just mention me in the #1143) or other data, as mentioned since this is very sporadic, I don't think I can create a reproducer, but I think I can increase the probability of encountering it

[angelozerr]

@E14 it is very hard to reproduce it since this override of classpath with Quarkus deploiement (to retrieve Quarkus properties from deploiement JAR) reacts when library changed with some debounce. If you have no luck you encounter this so awfull performance problem.

We already tried to fix this issue with the PR #1080 but it degrades performance of Quarkus properties compute. I have some ideas but it is not trivial.

Let's me finishing #1188 and after that I will work on this issue.

[angelozerr]

@E14 I have a draft PR #1221 which should improve the Quarkus deployment support :

• Quarkus deployment should be done only if need (when application.properties dooes validation, when there is a completion on applictaion.properties, etc) and not on each classpath changes
• the compute of Quarkus deployment should be very fast because I resolve the only the Quarkus deployment artifact and dont resolve transitly the deployment artifact.

To reproduce your isse, you add a depndency in pom.xml and you refresh your pom.xml and you do that several times. You should see your issue. With my draft PR #1221, I cancel the Quarkus deployment process as soon as there is a classpath changes.

[angelozerr]

Indeed I'm working on #1221 but just for the test, so I think you can test it by downloading IJ Quarkus zip at https://github.com/redhat-developer/intellij-quarkus/actions/runs/6532134435 and followthe instruction at https://github.com/redhat-developer/intellij-quarkus#testing-the-ci-builds

It should be really nice if you could install it and give us feedback. Thanks!

[E14]

So, I did a quick test, I'm not sure this is connected, but I saw this after installing the plugin snapshot:

A restart of IntelliJ fixed it (it didn't go away within ~2 hours where I didn't use IntelliJ), but I'm not sure it's even related to the Quarkus plugin

Otherwise I was not able to exceed two processes, though I had one case where UI blocked me with message "reformatting code", but I'm not expecting this to be related to the Quarkus plugin either.

FYI/PS I'm loading two IntelliJ projects (ie. two separate IntelliJ window frames) with multiple maven modules each, I noticed previously that this issue gets worse the more modules are loaded and much worse with multiple projects

[fbricon]

Reading Maven projects... is a pure IJ task, Quarkus Tools should not impact it

[E14]

@fbricon I noticed the last few days that the "maven pom reload" button sometimes does not appear anymore, could that be a result of this change? I'm still running the build you gave me

[fbricon]

Not sure but I doubt it. We're not touching anything close to the Maven config. I also saw the button not show up, but it's been happening for a very long time for me. So unlikely this is caused by this particular fix.

BTW 1.29.0 has been released on the marketplace yesterday

[E14]

I noticed the last few days that the "maven pom reload" button sometimes does not appear anymore, could that be a result of this change?

@fbricon so, I monitored this for a while now, and this still happens with the release version, but not regularly.

The reason I suspected this change is because of this:

With my draft PR #1221, I cancel the Quarkus deployment process as soon as there is a classpath changes.

I was concerned that the "cancel" also prevents further actions, like marking the project as requiring a rebuild when that happens.

When this happens it also seems to affect other build items, eg. I deleted a test case (class), and afterwards it would still find the testcase when running the maven install lifecycle, only after manually running maven clean lifecycle would it not find that class anymore

[angelozerr]

@E14 I think it should be better to process the quarkus deployment only when project is imported and not when librairies changed.

I have implemented this project import listener for the auto create Quarkus run config, I will try it to use this listener for deployment.

[E14]

@angelozerr I'm sorry I just saw that I mentioned the wrong user previously

Anyway, I'm not certain that this change is the cause, this is much less researched than the original issue (I assume due to the overall much improved build process? 🙂👍) and I haven't had any luck debugging this, there are no error messages or other indicators, so unless you are certain -- Is there any way to enable debug logging? That could help me find out more.

Sadly, depending on what you change in the pom however, and what you need from it in the plugin, a pom change might very well require tooling reload (Quarkus version change, adding/removing Quarkus dependencies, shared dependency change, etc.)