FIX - Ci improvement - maven artifact caching and various tweaks

[Agnul97]

Brief description of the PR.
This PR improves the CI process, with various tweaks and re-introducing the maven artifacts cache mechanism.

Related Issue
This PR addresses #3688

Description of the solution adopted
I'll present the changes made with a list that covers the different "categories" of improvements, sorted according to the order of implementation of each of them:

1. Duplicated code removal
   I've noticed that a lot of code for job definitions was repeated in the kapua-ci.yaml file, especially in the case of jobs that executes the different cucumber tests of the project. This redundancy of code causes difficulty in the maintainability of it, for example, a change that affects the jobs has to be repeated for each of them, with the difficulties associated with this. For example, I noticed that some of these jobs were using the same GitHub Actions but with different versions. To solve this I used the GitHub composite Actions https://docs.github.com/en/actions/creating-actions/creating-a-composite-action, defining in a separate file the template of these jobs (in the action.yaml file under the "runTestsTaggedAs" folder) that can be "specialized" with the proper tag of cucumber tests to run. To this end, the job definitions inside the Kapua-ci.yaml file re-use this template passing the different tests tag.

2. Execution of cucumber tests in the proper maven projects
   In a past issue related to tests for Kapua, I noticed that cucumber tests reside in a very small set of folders (or, in other words, maven projects). I noticed that, if I ran tests with a naive "mvn verify " a lot of time was spent unnecessarily in the "traversal" of a lot of maven projects not containing tests at all. I wanted to solve this waste of time by launching the maven tests execution only in the projects containing the tests. So, instead of the naive "mvn verify..." I used something like "mvn verify -pl list of projects containing cucumber tests". This, obviously, works assuming that maven artifacts and docker images needed by those tests are present in the environment. To retrieve the project containing tests I used the command "grep -l -r --include *.feature ."

3. Removal of jobs executing tests no more present in the code-base
   I noticed that the jobs triggerServiceIntegration , triggerServiceIntegrationbase , triggerService were trying to launch tests that are no longer present in the code base. Discussing with @Coduz we concluded that probably they may have been moved to the job tests suites, in any case, I will conduct an investigation to try to understand where they have been moved (or lost). I removed those 3 jobs considering that they were using machine time uselessly.

4. Maven Artifacts Caching
   Before this PR, there was an attempt to cache artifacts, exploiting the GitHub Action "actions/cache" but it was incomplete and actually not used. This Action was doing a save-restore for the cache but, despite this, each different job was executing a full build and in this way not re-using the cached artifacts. Other than this, the key parameter used to configure the caching mechanism (precisely key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} ) did not limit the scope of the cache to the specific workflow run but rather to the changes made in the root pom file. The cause of this was that, to propose an example, one could have done a series of commits to the same PR, without modification to the pom file BUT with modifications in the code, and the CI process for each of them would retrieve the same artifacts cache, without noticing changes in the code. To this end, The key parameter has been changed to scope the cache as correctly needed.
   This caching allows re-using artifacts in each job. Therefore, a full build is no longer necessary before running the cucumber tests. Rather, it is sufficient to build the docker images only.

The caching of docker images would be another good idea to prevent image re-building for each job. This task presents other difficulties and trade-offs and will be addressed in a separate PR.

Any side note on the changes made
The execution of tests with the specification of fixed set of maven projects creates the necessity of future modification of the set, if other tests are added in different maven projects. This is the trade-off for the speed-up obtained with the point 2 of the presented list. I think that this won't be a big concern if we consider the fact that the same is, at the moment, also true for the cucumber tags used in the different jobs (one could add tests with different tags and has to remember to add the specific job for it). In other words, a dependency on the kapua-ci file is already present for test addition and removal.

FINAL NOTES ON THE CORRECTNESS OF THIS PR
I considered from the start the requirement to maintain here the same tests that are run in the old CI process. In this regard, I verified manually that the new CI executes the same tests as the old one (and, also, the tests that are actually present in the code base). I report here the numbers of scenarios for each cucumber tag that I found in the code base and that are executed with this new CI

@brokerAcl: 59
@test-tag: 7 
@broker:  25
@device: 7
@devicemanagement: 22
@connection: 11 
@datastore: 56
@user : 30
@userIntegrationBase: 56
@userIntegration : 38
@security: 152 
@jobs,scheduler: 85
@jobsIntegrationBase: 82
@jobsIntegration: 8
@account,translator: 131
@jobEngineStepDefinitions: 5
@jobEngineStartOfflineDevice: 30
@jobEngineStartOnlineDevice: 25
@jobEngineRestartOfflineDevice: 30
@jobEngineRestartOnlineDevice: 14
@jobEngineRestartOnlineDeviceSecondPart: 12
@jobEngineServiceStop: 12
@role,group: 86
@deviceRegistry: 333
@endpoint: 66

[dseurotech]

Can this be removed completely? Having a retry plugin with attempts=1 just adds confusion

[Agnul97]

I maintained it thinking that the timeout mechanism of 45 mins could be useful for some reason. I would wait what Riccardo has to say to this regard

[riccardomodanese]

If there is no ather way (or action) to handle the timeout of a single job I prefer to keep the retry action.
Did you check the job 'timeout-minutes' parameter: timeout-parameter

[Agnul97]

I introduced the job timeout parameter, substituting the retry GitAction. Learning the provided doc. I understood that this has an equivalent result

[codecov]

Codecov Report

Merging #3785 (d676f58) into develop (fd877d7) will decrease coverage by 26.02%.
The diff coverage is 0.00%.

❗ Current head d676f58 differs from pull request most recent head e227c29. Consider uploading reports for the commit e227c29 to get more accurate results

@@              Coverage Diff               @@
##             develop    #3785       +/-   ##
==============================================
- Coverage      49.19%   23.17%   -26.02%     
  Complexity        26       26               
==============================================
  Files           1865     1866        +1     
  Lines          35275    35284        +9     
  Branches        2786     2782        -4     
==============================================
- Hits           17352     8178     -9174     
- Misses         17003    26794     +9791     
+ Partials         920      312      -608     

Impacted Files	Coverage Δ
...pse/kapua/job/engine/commons/logger/JobLogger.java	0.00% <0.00%> (ø)
...resources/v1/resources/UserCredentialFiltered.java	0.00% <0.00%> (ø)
...ice/endpoint/internal/EndpointInfoServiceImpl.java	0.00% <0.00%> (-80.30%)	⬇️

... and 563 files with indirect coverage changes