Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not fail on invalid extracted requirements #1429

Merged
merged 2 commits into from
Nov 12, 2024

Conversation

AyanSinhaMahapatra
Copy link
Member

@AyanSinhaMahapatra AyanSinhaMahapatra commented Nov 1, 2024

We were getting pipeline failures on wrongly parsed extracted requirements in someplaces:

'npm:wrap-ansi@^7.0.0' is not a valid <class 'univers.versions.SemverVersion'>

Traceback:
  File "/opt/scancodeio/aboutcode/pipeline/__init__.py", line 199, in execute
    step(self)
  File "/opt/scancodeio/scanpipe/pipelines/populate_purldb.py", line 48, in populate_purldb_with_discovered_dependencies
    purldb.populate_purldb_with_discovered_dependencies(
  File "/opt/scancodeio/scanpipe/pipes/purldb.py", line 364, in populate_purldb_with_discovered_dependencies
    unresolved_packages = get_unique_unresolved_purls(project)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/scancodeio/scanpipe/pipes/purldb.py", line 317, in get_unique_unresolved_purls
    vers = range_class.from_native(extracted_requirement)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/scancodeio/.venv/lib/python3.12/site-packages/univers/version_range.py", line 428, in from_native
    comparator=comparator, version=vrc(version_constraint)
                                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "<attrs generated init univers.versions.Version>", line 7, in __init__
    self.__attrs_post_init__()
  File "/opt/scancodeio/.venv/lib/python3.12/site-packages/univers/versions.py", line 87, in __attrs_post_init__
    raise InvalidVersion(f"{self.string!r} is not a valid {self.__class__!r}")
  • We should not fail the entire pipeline here, we were already catching InvalidVersionRange exceptions, but we also need to catch InvalidVersion exceptions here
  • Also adding project errors on invalid versions/version ranges as these can be package manifest parsing bugs.

scanpipe/pipes/purldb.py Outdated Show resolved Hide resolved
* Also add project errors on invalid versions/version ranges as
  these can be package manifest parsing bugs.

Signed-off-by: Ayan Sinha Mahapatra <[email protected]>
scanpipe/pipes/purldb.py Outdated Show resolved Hide resolved
Signed-off-by: Ayan Sinha Mahapatra <[email protected]>
@tdruez tdruez merged commit 0f868a1 into main Nov 12, 2024
9 checks passed
@tdruez tdruez deleted the extracted-requirements-error branch November 12, 2024 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants