Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(appset): scmProvider generator filters not applying properly (#14420) #14465

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jjsiv
Copy link
Contributor

@jjsiv jjsiv commented Jul 11, 2023

Fixes #14420

This PR fixes multiple scmProvider filters not applying properly when using combinations of such as repositoryMatch + branchMatch or multiple filters, as described in the mentioned issue.

Note on DCO:

If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • The title of the PR conforms to the Toolchain Guide
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • Optional. My organization is added to USERS.md.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).
  • My new feature complies with the feature status guidelines.
  • I have added a brief description of why this PR is necessary and/or what this PR solves.

Please see Contribution FAQs if you have questions about your pull-request.

@codecov
Copy link

codecov bot commented Jul 11, 2023

Codecov Report

Attention: Patch coverage is 92.68293% with 3 lines in your changes missing coverage. Please review.

Project coverage is 49.52%. Comparing base (1600c03) to head (644587a).
Report is 786 commits behind head on master.

Files Patch % Lines
applicationset/services/scm_provider/utils.go 92.68% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #14465      +/-   ##
==========================================
- Coverage   49.53%   49.52%   -0.01%     
==========================================
  Files         269      269              
  Lines       47222    47208      -14     
==========================================
- Hits        23392    23381      -11     
+ Misses      21530    21528       -2     
+ Partials     2300     2299       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@crenshaw-dev crenshaw-dev changed the title fix: AppSet scmProvider generator filters not applying properly fix(appset): scmProvider generator filters not applying properly Aug 24, 2023
@crenshaw-dev crenshaw-dev changed the title fix(appset): scmProvider generator filters not applying properly fix(appset): scmProvider generator filters not applying properly (#14420) Aug 24, 2023
@cyrus-mc
Copy link

This PR is a few months old. What is the next step to getting review and approval from the Argo team?

@crenshaw-dev
Copy link
Member

@cyrus-mc the prerequisites are done, it's down to review time at this point. Contributions currently exceed review capacity.

@alexelent-carta
Copy link

@crenshaw-dev bump 🙏

@saada
Copy link

saada commented Oct 5, 2023

bump 🙏🏽

@romancin
Copy link

Please review this one. We are having the same problem, which right now makes multi-repo architecture with discovery very difficult :(

Thank you!

@MaxFlandersLaerdal
Copy link

Would really love to see this fix get in!

@aarthjo
Copy link

aarthjo commented Oct 27, 2023

Good morning everyone,

It's also an urgent fix for us ! Thanks community

Signed-off-by: Jedrzej Kotkowski <[email protected]>
@jjsiv jjsiv requested a review from a team as a code owner November 24, 2023 20:09
Comment on lines +32 to -43
// FilterTypeRepo are filters that apply to the repo itself (name, labels)
FilterTypeRepo bool
// FilterTypeBranch are filters that apply to the repo content (path, branch)
FilterTypeBranch bool
}

// A convenience type for indicating where to apply a filter
type FilterType int64

// The enum of filter types
const (
FilterTypeUndefined FilterType = iota
FilterTypeBranch
FilterTypeRepo
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it's a good idea to use bools instead of just using the existing typed field.

What is the reasoning for this refactor?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FilterType would allow the filter to be of only a single type, but in reality it can be of both types - hence splitting it into two bools

@mtavaresmedeiros
Copy link

bump

@jjsiv jjsiv requested a review from jannfis March 10, 2024 20:23
@romancin
Copy link

romancin commented Mar 18, 2024

Any news on this one? It is killing GitHub repository auto-discovery every hour because of API rate limit :(

@white-hat
Copy link

Without this fix scmProvider is unusable for larger orgs, could you please prioritize this PR?

@aburan28
Copy link

Any update on this PR?

@jbaggs62
Copy link

Bump: this pr fixes a critical issues.

@jbaggs62
Copy link

jbaggs62 commented Oct 1, 2024

If you are following this thread the newest version of Argo release, had a similar fix and we have seen our request go down.

@romancin
Copy link

romancin commented Oct 1, 2024

If you are following this thread the newest version of Argo release, had a similar fix and we have seen our request go down.

Where? I can’t see references to this problem on the latest releases.

@akloss-cibo
Copy link

I didn't see any reference to improving this either, but I did update from 2.12.3 to 2.13.0-rc1 and we do see the rate improve quite a bit. This is our internal tracking of our github API quota:
image

It's still a lot and if you're not very careful will eat up your quota. (We have a hack in place where we keep the applicationset controller down to 0 replicas 80% of the time.)

@jbaggs62
Copy link

jbaggs62 commented Oct 1, 2024

If you are following this thread the newest version of Argo release, had a similar fix and we have seen our request go down.

Where? I can’t see references to this problem on the latest releases.

It’s in the release notes for 1568165 on 12.2.4
Fix on perpetual appset reconciliation

@romancin
Copy link

romancin commented Oct 2, 2024

If you are following this thread the newest version of Argo release, had a similar fix and we have seen our request go down.

Where? I can’t see references to this problem on the latest releases.

It’s in the release notes for 1568165 on 12.2.4 Fix on perpetual appset reconciliation

Great, thank you very much! As this fix is present in 2.12.4, let's try it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Using multiple scmProvider filters doesn't work properly and causes unexpected behaviour