Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Community plugins #5610

Merged

Conversation

davidmirror-ops
Copy link
Contributor

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Signed-off-by: davidmirror-ops <[email protected]>
@davidmirror-ops davidmirror-ops added the rfc A label for RFC issues label Jul 31, 2024
Copy link

codecov bot commented Jul 31, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 36.30%. Comparing base (30d3314) to head (a72533b).
Report is 107 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5610      +/-   ##
==========================================
+ Coverage   35.90%   36.30%   +0.40%     
==========================================
  Files        1301     1305       +4     
  Lines      109419   110019     +600     
==========================================
+ Hits        39287    39943     +656     
+ Misses      66035    65920     -115     
- Partials     4097     4156      +59     
Flag Coverage Δ
unittests-datacatalog 51.37% <ø> (ø)
unittests-flyteadmin 55.59% <ø> (+1.85%) ⬆️
unittests-flytecopilot 12.17% <ø> (ø)
unittests-flytectl 62.21% <ø> (-0.07%) ⬇️
unittests-flyteidl 7.12% <ø> (+0.03%) ⬆️
unittests-flyteplugins 53.35% <ø> (+0.03%) ⬆️
unittests-flytepropeller 41.89% <ø> (+0.13%) ⬆️
unittests-flytestdlib 55.37% <ø> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: davidmirror-ops <[email protected]>

## 6 Open questions

- Does this apply to Agents?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should. many people contribute agents as well

rfc/system/0008-community-plugins.md Show resolved Hide resolved

- There's a management overhead associated with this initiative:
- For every new plugin contributor, we may need to update `CODEOWNERS`
- CI configuration changes in flytekit (probably a one-time change)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'll need to rearrange a few things, but yeah, definitely a one-time small change.

rfc/system/0008-community-plugins.md Show resolved Hide resolved
Signed-off-by: davidmirror-ops <[email protected]>
- Create a `community` folder under `flytekit/plugins` and keep releasing the plugins in that folder as separate `pypi` packages.
- Configure CI to only run tests on `plugins/community` when there are changes.
- Keep releasing plugins alongside flytekit.
- Update [CODEOWNERS](https://github.com/flyteorg/flytekit/blob/master/CODEOWNERS) for the `plugins/community` folder, enabling contributors to merge community plugins (it requires granting Write permissions to contributors on `flytekit`).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Update [CODEOWNERS](https://github.com/flyteorg/flytekit/blob/master/CODEOWNERS) for the `plugins/community` folder, enabling contributors to merge community plugins (it requires granting Write permissions to contributors on `flytekit`).
- Update [CODEOWNERS](https://github.com/flyteorg/flytekit/blob/master/CODEOWNERS) for the `plugins/community` folder so that plugin authors are tagged for PR reviews.
- Enable contributors to merge community plugins (it requires granting Write permissions to contributors on `flytekit`).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's talk about this point today. Will every plugin author automatically get the permission to approve and merge PRs in the entire repo? Github's permissions model is a bit limiting for us here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revisiting this: how the contribution process would look like? Shouldn't be the existing CODEOWNERS (a.k.a. flytekit maintainers) who review and merge a new community plugin? In such case, we wouldn't need to update CODEOWNERS for every new plugin contribution.
Otherwise with the Github permission model, any new plugin contributor would have to be granted write permissions to the entire flytekit repo.
cc @eapolinario

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't be the existing CODEOWNERS (a.k.a. flytekit maintainers) who review and merge a new community plugin?

Yes, a new plugin should be merged by existing maintainers. The question for me is rather what happens after a plugin has been merged and needs to be maintained by the authors.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What we talked with Eduardo today is that, considering the implication of CODEOWNERS, we would go the regular contribution path: plugin authors would have non-binding approval rights on changes to their plugins, and flytekit maintainers issue the final approvals and merge. We'd also keep releasing community plugins with flytekit, even if there are no changes. Does that make sense?

Comment on lines 56 to 57

- Does this apply to Agents?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Does this apply to Agents?
-

fg91 and others added 2 commits August 29, 2024 20:43
Signed-off-by: Fabio M. Graetz, Ph.D. <[email protected]>
Signed-off-by: davidmirror-ops <[email protected]>
@davidmirror-ops
Copy link
Contributor Author

Contributors sync notes: @davidmirror-ops to explore if Triage perms would help give plugin authors non-binding approval rights, with maintainers retaining the binding approval and review. Otherwise plugin authors would have to (non-binding) approve via comments.

Signed-off-by: Fabio M. Graetz, Ph.D. <[email protected]>
Signed-off-by: Fabio M. Graetz, Ph.D. <[email protected]>
Signed-off-by: Fabio M. Graetz, Ph.D. <[email protected]>
Copy link
Member

@fg91 fg91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davidmirror-ops do you agree with merging this and this proposal? Then it LGTM.

Co-authored-by: Fabio M. Graetz, Ph.D. <[email protected]>
Signed-off-by: David Espejo <[email protected]>
@davidmirror-ops davidmirror-ops enabled auto-merge (squash) September 26, 2024 19:57
@davidmirror-ops davidmirror-ops merged commit bdaf79f into flyteorg:master Oct 18, 2024
50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rfc A label for RFC issues
Projects
Status: Accepted
Development

Successfully merging this pull request may close these issues.

5 participants