Support per-repo (or repo type?) configuration

[jeefy]

This is somewhat related to #66 but warrants a new Issue/Thread

Many projects have different repos for different purposes. clomonitor should expand the db schema so repos can be flagged as governance, docs, or code repos.

For example, Knative has several repos:

• Community -- A governance repo that holds a lot of policy/procedure info. This should be the primary repo, but it would have a different license (CC 4.0) since it's more of a "Docs" repo. It also is going to fail any sort of "recent release" check since it's not a code repo.
• Eventing -- A code repo that should have an Apache 2.0 License and a "recent release" check
• Service -- A code repo that should have an Apache 2.0 License and a "recent release" check
• Docs -- A docs repo that should have a CC 4.0 License check

If we can decouple the idea of "primary/secondary" repos to instead allow a repo to be flagged as one or more of these categories, I think it will better match many of the projects that exist or will soon be onboarded.

[tegioz]

At the moment we support multiple repositories kinds (primary and secondary) and for each kind we run a different set of checks that can be configured separately. Some context about where the primary and secondary repository kinds come from: #15 (comment), #15 (comment), #15 (comment).

We can add more repositories kinds, that's not a problem. However, we need to keep in mind that the more we have with more configurable variants, the more complexity we'll introduce producing reports, scoring them and specially combining repositories scores to produce a project's final score with homogenized results.

I'd suggest we keep track of how often situations like this occur before making important changes that would affect all projects. Once we have collected more feedback and use cases we can decide if it's better going with the kinds we have, we need to add more or we handle this in a completely different way. We'll be happy to refactor whatever is needed 🙂

I think in many cases the problem described in #66 can be addressed nicely with some headers in the README file, which also makes the information more discoverable by users. We also support .github repos for the default community health files. Some projects' repositories may not need to be added to CLOMonitor at all, as with a reference to some key docs from the main one may be enough.

On a side note about KNative, it looks like neither the library we are using to detect licenses nor Github are detecting it properly. Maybe it's because the LICENSE file contains the text of two licenses (CC and Apache), but not sure. CC is not on our list of approved licenses either (that affects the approved license check), but we could change that.

[jeefy]

Zoning in on the License issue, that's where things get somewhat complex.
CC4.0 is the license we want projects to use only for their docs
Their code should fall under one of the existing licenses you have already (like Apache 2.0)

Because of this split-brained scenario, I'm personally trying to push for the existence of a LICENSE file as well as a LICENSE-DOCS file.

This has been discussed previously (cncf/project-template#12) and I think it's the ideal path forward. Likewise it should also simplify the license checks.

[tegioz]

Because of this split-brained scenario, I'm personally trying to push for the existence of a LICENSE file as well as a LICENSE-DOCS file.

Cool, that'd be nice 👍

[chalin]

Adding my 2 cents: I'd vote for more repo types than just primary and secondary, such as code, docs, and community(?). /cc @nate-double-u

[duglin]

Just brainstorming... but do you want to get into the mode of continually needing to define new repo types as new ones come along? Would it make more sense to simply let the repo owner specify (somehow) that they don't have/need a certain type of artifact? I would think that the most important thing is that the repo owner went thru the thought process about each one and explicitly said "we don't need this for our stuff" rather than trying to fit into CLOMonitor's definition of what type of repo it might be?

[tegioz]

You bring up a great point @duglin. Hopefully we'll be able to come up with some kinds that cover most cases. But it could be a problem, you are right.

We actually provide a way to declare checks exemptions per repository, but IMHO it should only be used in some exceptional cases, and not as a way of configuring CLOMonitor. The main problem I see with encouraging projects to go this way is that it could be misused easily and some important checks could be left out, reducing the utility of the tool.

[tegioz]

@cynthia-sg and I have started working on this issue today. Depending on the approach we take, quite a few changes may be required and we'd like to get on the right track with this as soon as possible in preparation for an upcoming 1.0 release.

At the moment, we support two repository kinds, primary and secondary, and each repo must use one of them.

We were considering how it would look like switching to three: community, code and docs, as suggested in this issue. However, instead of each repo having a single kind, multiple kinds could be assigned to a repo.

Even though the current model does not fit well in some cases, it's working fine for most projects. Many of them have a single repository that covers all CLOMonitor checks, so in those cases we could flag a single repo as community + code, for example.

A possible distribution of the checks per kind could be:

• Kind: code
  • Documentation / Changelog
  • Documentation / Contributing
  • Documentation / Maintainers
  • Documentation / Readme
  • License
  • License / Approved
  • License / Scanning
  • Best practices / Artifact Hub badge
  • Best practices / DCO
  • Best practices / OpenSSF badge
  • Best practices / Recent release
  • Security / SBOM
  • Security / Policy
• Kind: community
  • Documentation / Adopters
  • Documentation / Code of conduct
  • Documentation / Contributing
  • Documentation / Governance
  • Documentation / Readme
  • Documentation / Roadmap
  • Documentation / Website
  • Best practices / Community meeting
  • Best practices / Slack presence
  • (?) Security / Policy
  • Legal / Trademark disclaimer
• Kind: docs
  • Documentation / Readme
  • License
  • License / Approved

Any thoughts that can help shaping this are very welcome! 🙂

/cc @caniszczyk

[tegioz]

Hi! 👋

We've just added support for check sets. Check sets replace the previous primary and secondary kinds. Unlike kinds, multiple check sets can be applied to a single repository. At the moment the following check sets are supported: code, code-lite, community and docs. For more details about what checks are part of each set please see the checks documentation.

All repositories have been updated with their corresponding check sets. If you notice that any of them doesn't have the correct check sets assigned please let us know and we'll gladly update it.

Lots of things have changed internally, including how scores are calculated and merged, so if you find something that's not working as expected please ping us 🙂

[duglin]

I can't seem to find the docs for where to specify the "check" value. Where is that?

Also, should there be a link from the main README ( https://github.com/cncf/clomonitor ) to the docs for how to setup a repo for CLOMonitor? The docs in the README focus on a CLOMonitor dev, not a user.

[duglin]

oh never mind - I see that it's set on your side.

I think my comment about docs for users is still valid though :-)