Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some proxys turn out to be NoneType after some time #2819

Closed
nikromen opened this issue Jul 19, 2023 · 6 comments
Closed

Some proxys turn out to be NoneType after some time #2819

nikromen opened this issue Jul 19, 2023 · 6 comments

Comments

@nikromen
Copy link
Member

nikromen commented Jul 19, 2023

package_proxy from client is used in packit's wait-for-copr script.

There's package_proxy used in the loop and perhaps in case of network error it changes itself to None (?!, PackageProxy is initialized once when calling Client, it shouldn't change itself randomly to None) even though there is a check in the script that package_proxy is not None (??!!)

2023-07-18 12:37:55.156 logging.py        INFO   No package with name python-copr-common in copr fedora-copr-copr-2818
2023-07-18 12:38:05.193 logging.py        INFO   No package with name python-copr-common in copr fedora-copr-copr-2818
2023-07-18 12:38:15.235 logging.py        INFO   No package with name python-copr-common in copr fedora-copr-copr-2818
2023-07-18 12:38:25.275 logging.py        INFO   Traceback (most recent call last):
2023-07-18 12:38:25.275 logging.py        INFO     File "/usr/bin/wait-for-copr", line 8, in <module>
2023-07-18 12:38:25.275 logging.py        INFO       sys.exit(wait_for_copr())
2023-07-18 12:38:25.276 logging.py        INFO                ^^^^^^^^^^^^^^^
2023-07-18 12:38:25.276 logging.py        INFO     File "/usr/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
2023-07-18 12:38:25.276 logging.py        INFO       return self.main(*args, **kwargs)
2023-07-18 12:38:25.276 logging.py        INFO              ^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-07-18 12:38:25.276 logging.py        INFO     File "/usr/lib/python3.11/site-packages/click/core.py", line 1055, in main
2023-07-18 12:38:25.276 logging.py        INFO       rv = self.invoke(ctx)
2023-07-18 12:38:25.276 logging.py        INFO            ^^^^^^^^^^^^^^^^
2023-07-18 12:38:25.276 logging.py        INFO     File "/usr/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
2023-07-18 12:38:25.277 logging.py        INFO       return ctx.invoke(self.callback, **ctx.params)
2023-07-18 12:38:25.277 logging.py        INFO              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-07-18 12:38:25.277 logging.py        INFO     File "/usr/lib/python3.11/site-packages/click/core.py", line 760, in invoke
2023-07-18 12:38:25.277 logging.py        INFO       return __callback(*args, **kwargs)
2023-07-18 12:38:25.277 logging.py        INFO              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-07-18 12:38:25.277 logging.py        INFO     File "/usr/lib/python3.11/site-packages/wait_for_copr/__init__.py", line 65, in wait_for_copr
2023-07-18 12:38:25.277 logging.py        INFO       built_version = copr_package_proxy.get(
2023-07-18 12:38:25.277 logging.py        INFO                       ^^^^^^^^^^^^^^^^^^^^^^^
2023-07-18 12:38:25.277 logging.py        INFO   TypeError: 'NoneType' object is not subscriptable

in logs above, the existence of python-copr-common pkg is checked a few times and then randomly the package_proxy happens to be NoneType

logs:
https://download.copr.fedorainfracloud.org/results/packit/fedora-copr-copr-2818/srpm-builds/06182105/builder-live.log.gz

@lsm5
Copy link

lsm5 commented Sep 20, 2023

podman team also hit this issue recently: https://github.com/containers/podman/actions/runs/6247273398/job/16959529269#step:6:16

@praiskup
Copy link
Member

That seems different:

   File "/home/runner/.local/lib/python3.10/site-packages/wait_for_copr/__init__.py", line 66, in wait_for_copr
    if release in (built_version := build["source_package"]["version"]):
TypeError: argument of type 'NoneType' is not iterable

The built version becomes None, not the proxy?

@praiskup
Copy link
Member

The original failing code is here.

@praiskup
Copy link
Member

praiskup commented Sep 20, 2023

Aha, I'm better reading the traceback @nikromen posted and TypeError: 'NoneType' object is not subscriptable is actually raised by None[], not None.get() (which would raise AttributeError: 'NoneType' object has no attribute 'get').

So there's not a bug "proxy becomes None". The problem is somewhere with the return value, and @lsm5 is right that these issues have likely the same roots...

@lsm5
Copy link

lsm5 commented Sep 20, 2023

@lsm5 is right that these issues have likely the same roots

TBH, I didn't look too much into it 😄 , only commented cause @nikromen sent me the link and the failure looked similar to what I noticed. Anyway, glad the issues are documented!

lsm5 added a commit to lsm5/podman that referenced this issue Sep 25, 2023
`wait-for-copr` is still very flaky and has failed more often than not.
Ref: fedora-copr/copr#2819

This change to the fcos GHA will allow nightly builds pulling in
whatever packages exist on podman-next at that time without depending on
wait-for-copr.

The commit id will still be recorded in podman version as well as the
image tag, so auditing is not affected with this change.

[NO NEW TESTS NEEDED]

Signed-off-by: Lokesh Mandvekar <[email protected]>
@praiskup
Copy link
Member

This should be resolved in the latest wait-for-copr script. Please reopen if you still see the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

No branches or pull requests

3 participants