Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"project update" will always do "Restoring original project mainline states" leading to crash for large projects #121

Open
gernot-h opened this issue Feb 5, 2025 · 2 comments · May be fixed by #122
Assignees

Comments

@gernot-h
Copy link
Collaborator

gernot-h commented Feb 5, 2025

#116 introduced a side effect. If you call "project update", it will now always update all project mainline states:

> poetry run capycli project update -id 50c... -url https://sw360.siemens.com -t <token> -oa -i project.sbom 

CaPyCli, 2.7.0 - Create or update a project on SW360

Loading SBOM file project.sbom
Updating project...
  3 releases in SBOM
  3 releases in project before update
  3 releases in project after update
  Restoring original project mainline states...

Done

In the current implementation, it will update each release's project mainline state individually. This leads to unnecessary SW360 API calls and for large projects, you will end up with API rate limiting timeouts:

  Restoring original project mainline states...
Traceback (most recent call last):
  File "/root/.cache/pypoetry/virtualenvs/distroclearing-9TtSrW0h-py3.11/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/distroclearing-9TtSrW0h-py3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 897, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/distroclearing-9TtSrW0h-py3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 897, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/distroclearing-9TtSrW0h-py3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 897, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  [Previous line repeated 2 more times]
  File "/root/.cache/pypoetry/virtualenvs/distroclearing-9TtSrW0h-py3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 887, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/distroclearing-9TtSrW0h-py3.11/lib/python3.11/site-packages/urllib3/util/retry.py", line 594, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='sw360.siemens.com', port=443): Max retries exceeded with url: /resource/api/projects/e4/release/0c (Caused by ResponseError('too many 500 error responses'))
@gernot-h gernot-h self-assigned this Feb 5, 2025
@gernot-h gernot-h changed the title "project create" will always do "Restoring original project mainline states" leading to crash for large projects "project update" will always do "Restoring original project mainline states" leading to crash for large projects Feb 5, 2025
@gernot-h
Copy link
Collaborator Author

gernot-h commented Feb 5, 2025

I think there are two issues to be addressed here:

  • CaPyCli shouldn't do these updates at all for a simple "project update". This code should only run for --copy-from situations
  • We should also check if CaPyCli really needs to update project mainline states individually, or if we can't do a batch update for all states in one API call.

@gernot-h
Copy link
Collaborator Author

gernot-h commented Feb 5, 2025

I was wrong. The project mainline states will always be overwritten with current code when it calls

   def update_project(self, project_id: str, project: Optional[Dict[str, Any]],
                       sbom: Bom, project_info: Dict[str, Any]) -> None:
        """Update an existing project with the given SBOM"""
        [...]
        data = self.bom_to_release_list(sbom)
        pms = self.get_release_project_mainline_states(project)
        try:
            [...]
            result = self.client.update_project_releases(data, project_id, add=self.onlyUpdateProject)  # type: ignore

This will replace existing releases with the list from the SBOM and this SW360 API call will reset all project mainline states.

The problem thus has nothing to do with project copy, it will happen on each project update. So a mechanism to preserve states is actually needed.

It seems there's however a more simple solution: The endpoint api/projects/<id>/releases allows to specify not only a list of releases when updating the project, but also their state (and relation). If we use this variant, we should be able to keep all data when updating.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant