Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve integration with nixpkgs metadata #100

Merged
merged 10 commits into from
Dec 22, 2023
Merged

Conversation

henrirosten
Copy link
Collaborator

@henrirosten henrirosten commented Dec 15, 2023

This change improves the way sbomnix reads nixpkgs metadata:

  • Adds command-line tool and python library nixmeta, which allows summarizing nixpkgs meta-attributes from the given nixpkgs version.
  • Start using local nix metadata cache generated with nixmeta based on the given sbomnix target. Additionally, if available, pre-populate the meta cache from an online database, maintained and daily-updated on github.

Made possible by the above changes, this PR removes the sbomnix command-line argument --meta: meta-information is now automatically added for all sbomnix SBOMs.

Also worth noting is that this PR changes sbomnix, nixgraph, vulnxscan, and nix_outdated so each tool allows specifying the target as Nix flake reference in addition to the earlier nix store path (which is still also supported).

Other smaller changes:

Once this PR is merged, we can close the issue: #62.

@henrirosten henrirosten force-pushed the nixmeta branch 12 times, most recently from f5aa312 to 0cbf813 Compare December 20, 2023 14:32
@henrirosten henrirosten changed the title Introduce nixmeta Improve integration with nixpkgs metadata Dec 21, 2023
@henrirosten henrirosten marked this pull request as ready for review December 21, 2023 11:08
@henrirosten henrirosten requested a review from a team December 21, 2023 11:08
Copy link
Contributor

@tervis-unikie tervis-unikie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Admittedly a quick glance, but nothing poked my eye.

@henrirosten
Copy link
Collaborator Author

Had to push another version due to missing sbomnix dependency: https://github.com/tiiuae/sbomnix/compare/692be07416a8070f108baf862046bf2f6b71e6b1..e210ed8c881f76a5900a8c9dc26e8a05885f177d

This went unnoticed before, because the dependecy was in devshell, but not in sbomnix package.

@henrirosten
Copy link
Collaborator Author

Added more logging in: https://github.com/tiiuae/sbomnix/compare/e210ed8c881f76a5900a8c9dc26e8a05885f177d..b137ee1c9bc64ac0e72cd6d4458740cec32a9a56 as I noticed in testing on larger targets (such as Ghaf), that the tools were suspiciously silent while evaluating and force-realising the target, which can take some time on larger build targets and might raise some concerns for the command-line user.

After adding those traces, the tools output some more info as to what is taking time.

@henrirosten henrirosten merged commit e02af45 into tiiuae:main Dec 22, 2023
3 checks passed
@henrirosten henrirosten deleted the nixmeta branch January 3, 2024 07:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants