Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add file source digest support #1914

Merged
merged 1 commit into from
Jul 5, 2023
Merged

Add file source digest support #1914

merged 1 commit into from
Jul 5, 2023

Conversation

wagoodman
Copy link
Contributor

With #1846 it became possible to express the digest of a scanned file (e.g. syft ./go.mod -o json can now support showing a digest in the .source section). This new feature was only added in the json schema but not wired up to start adding values -- this PR adds this wiring.

The default algorithm for file sources is sha-256, however, this is configurable with SYFT_SOURCE_FILE_DIGEST which can take one or more values (comma separated). This additionally expands the allowable digest algorithms in general for syft to include other sha224, sha384, and sha512. While working on this I attempted to move more file API surface area to under internal.

I've migrated the source-name and source-version configuration options to reside under the new source struct, which will break users depending on source-name and source-version. This does not affect users depending on the CLI option.

@wagoodman wagoodman requested a review from a team July 5, 2023 15:32
@github-actions
Copy link

github-actions bot commented Jul 5, 2023

Benchmark Test Results

Benchmark results from the latest changes vs base branch
goos: linux%0Agoarch: amd64%0Apkg: github.com/anchore/syft/test/integration%0Acpu: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz%0A                                                          │ ./.tmp/benchmark-00a2401.txt │%0A                                                          │            sec/op            │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   13.14m ±  2%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    780.3µ ±  4%25%0AImagePackageCatalogers/binary-cataloger-2                                   219.4µ ±  1%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   645.8µ ±  2%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              1.349m ±  2%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         103.1µ ±  1%25%0AImagePackageCatalogers/java-cataloger-2                                     14.15m ±  2%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     101.2µ ±  1%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       455.1µ ± 18%25%0AImagePackageCatalogers/nix-store-cataloger-2                                302.4µ ±  5%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   854.6µ ±  1%25%0AImagePackageCatalogers/portage-cataloger-2                                  515.7µ ± 19%25%0AImagePackageCatalogers/python-package-cataloger-2                           3.537m ±  2%25%0AImagePackageCatalogers/r-package-cataloger-2                                232.2µ ±  2%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   599.7µ ±  4%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             979.7µ ±  2%25%0AImagePackageCatalogers/sbom-cataloger-2                                     127.8µ ±  1%25%0Ageomean                                                                     669.2µ%0A%0A                                                          │ ./.tmp/benchmark-00a2401.txt │%0A                                                          │             B/op             │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   5.125Mi ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    205.3Ki ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                   30.18Ki ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   168.9Ki ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              405.3Ki ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         9.906Ki ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                     2.828Mi ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     8.594Ki ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       100.9Ki ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                49.15Ki ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   186.7Ki ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                  119.9Ki ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                           1.003Mi ± 0%25%0AImagePackageCatalogers/r-package-cataloger-2                                53.29Ki ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   180.9Ki ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             144.1Ki ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                     14.20Ki ± 0%25%0Ageomean                                                                     132.7Ki%0A%0A                                                          │ ./.tmp/benchmark-00a2401.txt │%0A                                                          │          allocs/op           │%0AImagePackageCatalogers/alpmdb-cataloger-2                                    87.75k ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                     4.182k ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                     830.0 ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                    3.002k ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                               6.338k ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                           281.0 ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                      39.88k ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                       228.0 ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                        1.404k ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                  895.0 ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                    4.079k ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                   2.269k ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                            16.44k ± 0%25%0AImagePackageCatalogers/r-package-cataloger-2                                  929.0 ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                    3.989k ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                              2.447k ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                       394.0 ± 0%25%0Ageomean                                                                      2.583k

.gitignore Show resolved Hide resolved
@wagoodman wagoodman added the enhancement New feature or request label Jul 5, 2023
@wagoodman wagoodman merged commit cfbb9f7 into main Jul 5, 2023
@wagoodman wagoodman deleted the add-source-file-digest branch July 5, 2023 17:47
spiffcs added a commit that referenced this pull request Jul 11, 2023
* main:
  feat: CLI flag for directory base (#1867)
  Fix CPE gen for k8s python client (#1921)
  chore: update iterations to protect against race (#1927)
  chore(deps): update bootstrap tools to latest versions (#1922)
  fix: Don't use the actual redis or grpc CPEs for gems (#1926)
  fix(install): return with right error code (#1915)
  Remove erroneous Java CPEs from generation (#1918)
  chore(deps): bump golang.org/x/net from 0.11.0 to 0.12.0 (#1916)
  Switch UI to bubbletea (#1888)
  fix: use filepath.EvalSymlinks if os.Readlink fails to evaluate the link (#1884)
  add file source digest support (#1914)
  chore(deps): update bootstrap tools to latest versions (#1908)
  chore(deps): bump golang.org/x/mod from 0.11.0 to 0.12.0 (#1912)
  chore(deps): bump golang.org/x/term from 0.9.0 to 0.10.0 (#1913)
  doc(readme): add installation section with scoop (#1909)
  Refactor source API (#1846)
  chore(deps): update bootstrap tools to latest versions (#1905)
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants