Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JBPM-10184] Build templates optimization #1600

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

albfan
Copy link

@albfan albfan commented Jul 25, 2023

RHPAM-4746

This change tries to avoid extra builds after a template is already build.

TODO: Still unsure if that could lead to errors if template is changed and new execution servers are registered

@albfan albfan changed the title Build optimization Build templates optimization Jul 25, 2023
@albfan albfan changed the title Build templates optimization [JBPM-10184] Build templates optimization Jul 25, 2023
@albfan
Copy link
Author

albfan commented Jul 25, 2023

Add test on jbpm-wb-kie-server/jbpm-wb-kie-server-backend/src/test/java/org/jbpm/workbench/ks/integration/KieServerIntegrationServerTemplateTest.java

@mareknovotny
Copy link
Member

jenkins do fdb

@mareknovotny
Copy link
Member

ok to test

@mareknovotny
Copy link
Member

@albfan are you adding the required test? Please respond/add it until Friday EOD

@albfan
Copy link
Author

albfan commented Aug 21, 2023

I'm working on that test

@cristianonicolai cristianonicolai requested review from nmirasch and removed request for cristianonicolai August 30, 2023 08:16
@mareknovotny
Copy link
Member

jenkins retest this please

@mareknovotny
Copy link
Member

jenkins do fdb

Copy link
Member

@gmunozfe gmunozfe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@albfan Some questions about changing order of operations.
Also a test would be helpful to check that the optimization is working as expected and coverage is apropiated

else {
logger.info("Server {} connected!", serverInstance);

serverInstancesById.put(serverInstance.getServerInstanceId(), serverInstance);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the previous code before this refactoring, the put operation was after buildClientsForServer, not sure why it was moved before here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the purpose of the PR is to avoid calling buildClients if the server is already in the map. However, as mentioned in my comment, this should be done with computeIfAbsent

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pushing a refactor with minimal changes so we can identify what changes, I think Gonzalo is right, put is changed, still not sure if that makes any difference

@mareknovotny
Copy link
Member

any progress on updates after review @albfan ? Also @nmirasch could you review this too?

@mareknovotny
Copy link
Member

ping @albfan, are you on this PR remaining test?

Copy link
Contributor

@fjtirado fjtirado left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, but please check refactor comment for optimal implementation

@albfan
Copy link
Author

albfan commented Feb 21, 2024

I think I can describe now what is this case for:

Under openshift all kie server are under same location (replicas acts behind a smart router). If you start a kie server with 3 replicas you will have 3 exact request of onServerInstanceConnected. if serverInstancesById put is set after the build, you will get 3 builds.

Setting put first you filter that. Probably same behaviour as a smart router.

That probably will mark no available endpoints if some kie server is shutdown (while there're still 2 kie servers alive)

Let me check if we allow duplicates on loadbalancer, at least with smart router, if no this is not correctly handled

@fjtirado
Copy link
Contributor

jenkins retest this

@albfan
Copy link
Author

albfan commented Feb 21, 2024

So this optimization is just a corner case to detect an ongoing task that will end on

    protected void indexServerInstances(ServerTemplate serverTemplate) {
        serverTemplate.getServerInstanceKeys().forEach(serverInstanceKey -> serverInstancesById.put(serverInstanceKey.getServerInstanceId(),
                                                                                                    serverInstanceKey));
    }
    ```
    
which just happens after all kjars are build, but that will ignore any possible error deploying the template

All request will come from same url, and will not detect ongoing builds
@albfan
Copy link
Author

albfan commented Feb 22, 2024

@gmunozfe I keep the current behaviour (put after build) probably unneeded if indexServerInstances is reached, but if no build is triggered probably still needed. I think in case of error building this is wrong, but that's different story

@fjtirado I added a tentative property to deal with this corner case (repeated builds from same url) this can only happen with customer with kie servers under load balancer, and starting at same time several kie servers (like openshift does with replicas)

Still using this serverInstancesById looks wrong, as probably what we want to detect is we already started a build for that (like a cache of ongoing builds) to allow all the detection of fails to continue

return clients;
});

serverInstancesById.put(serverInstance.getServerInstanceId(),
Copy link
Contributor

@fjtirado fjtirado Feb 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line should be removed. serverInstance will be added to the map when this function returns

Comment on lines +251 to +254
if (Boolean.parseBoolean(System.getProperty("org.kie.server.startup.detectOpenshiftLoadBalancer", "false"))) {
serverInstancesById.put(serverInstance.getServerInstanceId(),
serverInstance);
}
Copy link
Contributor

@fjtirado fjtirado Feb 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (Boolean.parseBoolean(System.getProperty("org.kie.server.startup.detectOpenshiftLoadBalancer", "false"))) {
serverInstancesById.put(serverInstance.getServerInstanceId(),
serverInstance);
}
if (Boolean.getBoolean("org.kie.server.startup.detectOpenshiftLoadBalancer")) {
return serverInstance;
}

serverInstance);
return serverInstance;
});
if (serverInstance != serverInstanceKey) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets do that differently, rather than tracing "isAlreadyRegistered", add a trace when it is registered. If this new registering trace is not present, it can be assumed it has not been registered

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants