Might need to use more than just Clomonitor

[cprivitere]

Many subprojects are missing from Clotributor because it uses clomonitor to pull in CNCF projects which skips over most of the sub-projects.

What should be the the route to take here, have it so every project needs to manually request it be added to clomonitor?

See these two tickets for reference when I asked for some subprojects to be added:
cncf/clomonitor#1503
cncf/clomonitor#1504

[tegioz]

Hi @cprivitere!

(same maintainer here 😉)

The reason why CLOTributor relies on the same data files as CLOMonitor is that the target projects are mostly the same for both. We could separate them easily, as we only use a reference to the data file hosted in the CLOMonitor repository. But then it'd be needed to maintain both, which would be confusing (and more work) for contributors that are used to send PRs just to CLOMonitor. When a project/repo should only be used in one of them, it's possible to indicate this in the data file by using the exclude field (i.e. exclude CLOMonitor or CLOTributor).

As I mentioned in the other issues, the target are mainly CNCF (or other foundations) projects, not including all subprojects. However, given that we've included other Kubernetes subprojects for the Security Slam event, I think it'd be OK to accept some more. But we need to be careful as some projects (like Kubernetes) have literally hundreds of repositories across multiple organizations, and some changes to CLOTributor might be needed if we wanted to include them all to avoid reaching the GitHub API rate limits.

Maybe I didn't explain myself well in the other issues, but I tried to invite you to add those projects 😇 Although we maintain some of those data files, it's very common to receive PRs to include new projects or repositories (one example from today). In fact, contributors will often do a better job than us adding them as they know best which check set suits better each repository, or include only the repositories they'd like to be tracked.

You could add the projects/repositories you're interested in directly in a single PR. And soon after the PR is merged, both CLOMonitor and CLOTributor will pick up the changes and start processing the new projects.

Hope this helps 🙂

[cprivitere]

Thanks for the explanation. I should probably add some context to make it clear where I'm coming from.

I just heard about clotributor yesterday from Bob Killen and Jeffrey Sica on the Equinix Demo Days stream. My understanding coming out of that was that clotributor was a tool to get folks involved in CNCF related projects by showing them all the various good first issues.

I then came over here and noticed a bunch of missing projects (like the ones Equinix has donated to CNCF). I figured that meant something was overlooked. It now sounds like instead I misunderstood what the project is trying to do, and you're purposefully limiting the scope both to keep the site running well and (perhaps?) limit just how many things show up for folks to keep them from getting overwhelmed?

Before I just start yeeting projects into the data files, I'd like to make sure I'm not actually making things worse by doing so.

[tegioz]

No worries! And thank you for the context, it helps 🙂

You got it completely right! Our target projects (for the CNCF foundation) are these ones, as we try to keep them in sync regularly (until we start feeding from the landscape data now that we've migrated to v2 😉). One of our intentions is to keep things balanced among projects and promote all of them a bit. If we were to add all Kubernetes repositories, the full CLOTributor feed would be all about Kubernetes, and the other projects would be hardly noticed.

But adding a few Kubernetes subprojects shouldn't be a problem 🙂

[tegioz]

Will close this issue for now @cprivitere, please feel free to reopen if needed 🙂