Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: combine integration tests #8253

Merged
merged 5 commits into from
Aug 27, 2023
Merged

ci: combine integration tests #8253

merged 5 commits into from
Aug 27, 2023

Conversation

mhofman
Copy link
Member

@mhofman mhofman commented Aug 25, 2023

refs: #7629

Description

This combines all longer integration type tests into a single workflow.

It also restores the logic to avoid running these tests on PRs before ready to merge.
It also restores duplicate checks to avoid re-running these tests on the exact same content.

It does not remove the github based concurrency limit for pull request, which makes these tests potentially cancel and re-start on the same content if the test was still in progress. I remember there were issues in the custom concurrent checks logic and I'll look into restoring that later.

This is the first step towards switching back to mergify.

Security Considerations

None

Scaling Considerations

Not production scaling, but this should help reining in how often these expensive CI jobs run.

Documentation Considerations

None for now

Testing Considerations

Will test the behavior on this PR

Upgrade Considerations

None

@mhofman mhofman enabled auto-merge August 25, 2023 20:40
@mhofman mhofman disabled auto-merge August 25, 2023 20:42
@mhofman mhofman enabled auto-merge August 25, 2023 20:49
@mhofman mhofman disabled auto-merge August 25, 2023 20:50
@mhofman mhofman added the force:integration Force integration tests to run on PR label Aug 25, 2023
@mhofman mhofman force-pushed the mhofman/combine-integration branch from 56c9034 to ccbd066 Compare August 25, 2023 21:45
@mhofman mhofman requested review from warner and raphdev August 25, 2023 21:46
@mhofman mhofman removed the force:integration Force integration tests to run on PR label Aug 25, 2023
@mhofman mhofman force-pushed the mhofman/combine-integration branch 2 times, most recently from 43190e4 to a139345 Compare August 25, 2023 22:48
@mhofman mhofman enabled auto-merge August 25, 2023 22:55
@mhofman mhofman force-pushed the mhofman/combine-integration branch 2 times, most recently from a188441 to b3f0271 Compare August 25, 2023 22:58
@mhofman
Copy link
Member Author

mhofman commented Aug 25, 2023

Ok I think this is actually ready for review now.

@warner I don't automatically expect a review, just more an FYI
@raphdev I'm not sure why the docker upgrade test was added to the "test all packages". It didn't depend on any of these other jobs, and doesn't seem like the right fit. Just want to make sure that moving it won't break the Datadog reporting.
@michaelfig you're still the most familiar with these CI config changes, please review :)

@mhofman mhofman force-pushed the mhofman/combine-integration branch from b3f0271 to 7de02d9 Compare August 25, 2023 23:56
@mhofman mhofman disabled auto-merge August 26, 2023 00:02
@mhofman mhofman force-pushed the mhofman/combine-integration branch from 7de02d9 to 9dc7bb5 Compare August 26, 2023 00:02
@mhofman mhofman added automerge:no-update (expert!) Automatically merge without updates and removed automerge:no-update (expert!) Automatically merge without updates labels Aug 26, 2023
@mhofman mhofman enabled auto-merge August 26, 2023 00:08
@mhofman mhofman disabled auto-merge August 26, 2023 00:20
@mhofman mhofman force-pushed the mhofman/combine-integration branch from 9dc7bb5 to 0d6ccef Compare August 26, 2023 00:23
@mhofman
Copy link
Member Author

mhofman commented Aug 26, 2023

Sorry for the churn. I had to remove the run to completion test (it's not compatible with the skip duplicate test, at least not without tweaks). I did add a couple checks for merge strategy and linear history (which we should be able to mark as required once this lands)

Copy link
Member

@michaelfig michaelfig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just one minor comment for consideration.

Thanks for doing this work... looking forward to the next review.

github.event.pull_request.draft == true ||
github.event.pull_request.base.ref != 'master' || (
contains(github.event.pull_request.labels.*.name, 'automerge:squash') ||
contains(github.event.pull_request.labels.*.name, 'automerge:merge') ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I'd like to rename automerge:merge to automerge:expert, in accordance with earlier discussions how it might otherwise be overused by people who may not know its nuances.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or automerge:no-update.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or automerge:manual-updates.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already updated the label description to indicate it's an expert option, and I'm hoping the new no-merge-commits check will help reduce it's usage (it should bump it off the queue if a merge commit is added to make the PR up to date)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of these I prefer the automerge:no-updates

@mhofman mhofman force-pushed the mhofman/combine-integration branch from 0d6ccef to e0eb08b Compare August 27, 2023 20:08
@mhofman mhofman linked an issue Aug 27, 2023 that may be closed by this pull request
@mhofman mhofman enabled auto-merge August 27, 2023 20:15
Copy link
Member

@michaelfig michaelfig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can speak on behalf of the DataDog test recording integration: it won't break since it's dynamically registered, but may possibly start logging to a different metric than before. As long as @raphdev is aware, I think this is safe to merge.

@michaelfig
Copy link
Member

As discussed, the local-npm failure is expected until #8250 lands.

@mhofman mhofman added this pull request to the merge queue Aug 27, 2023
Merged via the queue into master with commit 2ffcef2 Aug 27, 2023
73 of 74 checks passed
@mhofman mhofman deleted the mhofman/combine-integration branch August 27, 2023 20:57
mhofman added a commit that referenced this pull request Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

block autosquash/fixup commits
2 participants