Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Breaking syft | grype or merging syft code that breaks grype tests should warn team #1939

Open
willmurphyscode opened this issue Jul 13, 2023 · 2 comments
Labels
release relating to the release process of syft

Comments

@willmurphyscode
Copy link
Contributor

What would you like to be added:

The release of a new version of syft should exercise syft | grype and, if grype fails, warn the team that syft is ahead of grype, in the sense that the latest version of syft produces output the latest version of grype can't parse. The warning (e.g. slack alert to the team) should remind the team to promptly release grype so that the period where syft is ahead is very short.

Why is this needed:

#1935 is an example where we had customer impact because a syft release went out that the team didn't realize was incompatible with latest grype, and so syft | grype for latest syft and grype was broken for a couple of days.

@willmurphyscode willmurphyscode added the release relating to the release process of syft label Jul 13, 2023
@wagoodman
Copy link
Contributor

wagoodman commented Jul 13, 2023

Though failing CI in a PR isn't an option, adding a comment on the PR when it will break grype could be a good option in addition to your suggestion (in the vein of shift-left).

@willmurphyscode willmurphyscode changed the title Breaking syft | grype should warn team Breaking syft | grype or merging syft code that breaks grype tests should warn team Aug 28, 2023
@willmurphyscode
Copy link
Contributor Author

We discussed this issue again at retro, and the solution we were leaning towards was:

  1. New commits on main in syft should create (or update if existing) a PR to grype that bumps syft in grype to the new commit.
  2. Releases of syft should query for this PR and assert that its tests are is passing.
  3. There should be a good way to override the checks in step 2 if needed.
  4. When we release syft, we can then immediately merge this PR and release grype.

This way, regressions can be caught by grype's test suite, which is probably the right place for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release relating to the release process of syft
Projects
None yet
Development

No branches or pull requests

2 participants