Tests: 1175 ci refactor e2e

[bermuell]

Description

Closes: #1175

• Created github workflow e2e tests
• changed e2e tests for easier test selection (see -tc argument)
• dedicated jobs for each e2e test case on manual and nightly workflow

---

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

• Included the correct type prefix in the PR title
• Targeted the correct branch (see PR Targeting)
• Provided a link to the relevant issue or specification
• Reviewed "Files changed" and left comments if necessary
• Confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

• Confirmed the correct type prefix in the PR title
• Confirmed all author checklist items have been addressed
• Confirmed that this PR does not change production code

[MSalopek]

Approval!

Thanks for spending the time to refactor this!

[shaspitz]

My understanding is that the changed files manual-e2e.yml and nightly-e2e.yml are not used for CI. Rather automated-tests.yml is used for CI. The issue seems to mention improving CI reporting, which would be unchanged by this PR due to automated-tests.yml being unchanged. Am I misunderstanding something here?

[bermuell]

My understanding is that the changed files manual-e2e.yml and nightly-e2e.yml are not used for CI. Rather automated-tests.yml is used for CI. The issue seems to mention improving CI reporting, which would be unchanged by this PR due to automated-tests.yml being unchanged. Am I misunderstanding something here?

Reason is that the changes forseen for the automated tests (splitting E2E from UT, IT) were addressed by PR #1181 and introduction of cometmock based tests changed the need for splitting happy path test in respect of downtime (normal vs throttling).

[p-offtermatt]

LGTM. I pushed one change to the docstring of the short happy path.

I also think we might want to rename the short happy path to cometmock-happy-path or something like this, since it's getting longer and really the main distinction is that it avoids things incompatible with cometmock specifically. But this is not really that important imo, so I'm happy with this as-is, too