Skip to content

Commit

Permalink
refactor(api): refactor purl API query code (#2900)
Browse files Browse the repository at this point in the history
Converted PURL queries to package queries in `do_query()` to remove
unnecessary code. Querying the Datastore directly with PURLs gives the
same results as querying by package name, so we can simplify things by
treating PURLs like regular package queries after extracting the
`package/ecosystem/version`.

This will also resolve issue #2842 by rewriting the PURL-to-ecosystem
logic. For most ecosystems, we can get the name from `purl.type`, but
for Linux distributions, we need to use `purl.namespace`.
  • Loading branch information
hogo6002 authored Nov 28, 2024
1 parent 123cab3 commit 7496fb4
Show file tree
Hide file tree
Showing 14 changed files with 614 additions and 396 deletions.
20 changes: 19 additions & 1 deletion docker/worker/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions docker/worker/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ google-cloud-ndb = "==2.3.2"
google-cloud-storage = "==2.18.2"
PyYAML = "==6.0.2"
redis = "==5.2.0"
packageurl-python = "==0.16.0"
pygit2 = "==1.16.0"
requests = "==2.32.3"
jsonschema = "==4.23.0"
Expand Down
342 changes: 233 additions & 109 deletions gcp/api/fixtures/IntegrationTests_api_query_response.txt

Large diffs are not rendered by default.

7 changes: 5 additions & 2 deletions gcp/api/integration_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,10 @@ def test_query_unknown_purl_invalid_semver(self):
}),
timeout=_TIMEOUT)

self.assert_results_equal({}, response.json())
self.assert_results_equal({
'code': 3,
'message': 'Invalid PURL.'
}, response.json())

def test_query_semver_no_vulns(self):
"""Test queries by SemVer with no vulnerabilities."""
Expand Down Expand Up @@ -911,7 +914,7 @@ def test_all_possible_queries(self):
}
}, {}]

pkg_version = [{'package': {'version': '0.8.5'}}, {}]
pkg_version = [{'version': '0.8.5'}, {}]

commit = [{'commit': 'd374094d8c49b6b7d288f307e11217ec5a502391'}, {}]

Expand Down
3 changes: 2 additions & 1 deletion gcp/api/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion gcp/api/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ python = "^3.11"

google-cloud-ndb = "==2.3.2"
google-cloud-logging = "==3.11.3"
packageurl-python = "==0.16.0"
packaging = "==20.9"
requests = "==2.32.3"
grpcio = "==1.64.1"
Expand Down
Loading

0 comments on commit 7496fb4

Please sign in to comment.