Run all tests for new libraries versions checking at once

[dnestoro]

What does this PR do?

Implements: #672

Implementation details

• Instead of returning a flat list of new versions, the fetchExistingLibrariesWithNewerVersions task now returns versions grouped by library.
• The workflow then gets this map, and spawns a new job for each key (in this case, name) in the returned map.
• For each spawned job, we are executing the tests for that library with newer versions sequentially using this script.
• Once the script finds first failing version, it returns list of all successful versions and a single (first) failing version.
• Later we process those results to create a PR for all successful versions and a ticket per library that contains information about first failing version.

Note: I've changed the workflow trigger mechanism. From now on, it can only be triggered manually.

[vjovanov]

Why do we need this? What if I fork this repo?

[dnestoro]

Previously this workflow was scheduled for execution every two weeks, and therefore we didn't want this to be executed every two weeks on user's forks (we would spend their repo resources that way). We can remove it now if necessary, but I don't see a reason why this job should be executed anywhere else than in this repo.

[vjovanov]

Interesting, just leave it as a comment. Is this standard practice?

Will this prevent me from running this task manually on my fork, and should it?

[dnestoro]

Yes, that will prevent you from running this workflow on your fork (that is the purpose of this). As long as this is not a scheduled, but manually triggered job, we can remove this line. If we decide to make it scheduled at some point, I will keep this line.

[vjovanov]

Why not run this every week, after the initial manual run?

[dnestoro]

Why not run this every week, after the initial manual run?

Once we establish a quick way for fixing many tests at once, we can make this scheduled. Until then, running tests every week, while we only fixed 1 or 2 tests, will just tell us that we have same failures every week (while we spend so much resources doing that). So in my opinion, that is something for TODO list here. Once we have proper way of bulk fixing tests, we can easily make this job scheduled.

[vjovanov]

Why not save the information about the failing builds in the repo so we don't re-run them?

Then we can schedule every week to pick up the new versions that are passing.

[dnestoro]

Update:

• Added retry logic
• Instead of keeping information about previous failures in additional file, I used the fact that we create tickets when we have failures. So before we trigger new tests run for some library, we will check if there is an existing issue for that library and the first version we want to test (implementation). In case there is a ticket for the library mentioning that the failing version is the same as the first one we want to run again => we will skip the test. cc @vjovanov
• The workflow is now scheduled for execution once a week