-
Notifications
You must be signed in to change notification settings - Fork 86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve newline handling in collectors #458
Comments
@erikgb if you provide a reproducer, I can take a look. |
@iblancasa Thanks, I'll see if I can build a simple reproducer. |
With some help from a Rancher/k3s maintainer, I managed to capture the raw container logs on the node:
And that does not look correct, as the last line has a How did you try to reproduce? I am thinking this might happen if a pod logs without a newline on it's last line before terminating. |
I tried multiple ways. Like creating a pod that doesn't print any new lines and collects the output... the same but terminating the pod while printing the output... |
A reproducer will help a lot. |
What happened:
While debugging a flake issue in our e2e-test, we used collectors to harvest logs from relevant pods, we think the output could handle newlines (or actually missing newlines) better. Thanks a lot for adding this feature including docs! ❤️
The issue we were debugging, is job pods that are supposed to produce valid JSON output to stdout. For some reason, it does not do so occasionally, as it gets truncated. Here is an example of the relevant collector output:
The collector selector matches three single container pods. Two of them produce valid output, but for the container
pod/deployment-vuln-app-app-eccf8-52873-dnp82/scan-image
the logs are suddenly truncated in the middle of a JSON property, and the log ends without the standard newline. This results in kuttl logging the following line:And this kuttl log-line does not match the format and structure in the other lines.
What you expected to happen:
Kuttl logs in a consistent format, even if a container log isn't terminated by newline.
How to reproduce it (as minimally and precisely as possible):
I haven't tested this, but I would expect this to be reproducible by collecting a pod log not terminated by newline.
Anything else we need to know?:
Environment:
kubectl version
): 1.26.0kubectl kuttl version
): 0.15.0uname -a
):The text was updated successfully, but these errors were encountered: