Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

first line of non-datetimestamped text is dropped #287

Open
jtmoon79 opened this issue Apr 20, 2024 · 0 comments
Open

first line of non-datetimestamped text is dropped #287

jtmoon79 opened this issue Apr 20, 2024 · 0 comments
Labels
difficult A difficult problem; a major coding effort or difficult algorithm to perfect enhancement New feature or request P2 less important

Comments

@jtmoon79
Copy link
Owner

jtmoon79 commented Apr 20, 2024

Describe the bug

Given a text log file wherein the first X lines have no matching datetimestamp, s4 will not print those lines.

To Reproduce

Using logs/RancherOS-1.5.8/docker.log

$ s4 ./logs/RancherOS-1.5.8/docker.log
time="2022-10-09T21:33:15.541012931Z" level=info msg="Starting up"
time="2022-10-09T21:33:15.547531433Z" level=info msg="libcontainerd: started new containerd process" pid=1178
...

however, the file logs/RancherOS-1.5.8/docker.log has contents

crypto/rand: blocked for 60 seconds waiting to read random data from the kernel
time="2022-10-09T21:33:15.541012931Z" level=info msg="Starting up"
time="2022-10-09T21:33:15.547531433Z" level=info msg="libcontainerd: started new containerd process" pid=1178
...

This may be confusing to users. In this case, no datetime filter was passed so the user expects the entire file to be printed. But the first line is dropped.

Environment:

  • s4 version: 0.6.71

Additional context

This was an intentional design choice early-on. As a simplifying principle, the syslog log message is considered a log message wherein the first line contains the datetimestamp. Any different presumption means processing some ad-hoc text log files with multi-line log messages could be too difficult or impossible to process.
One additional reason to drop the first X lines without a datetimestamp, is it is unknown when those first lines of text were inserted, e.g. the message crypto/rand: blocked for 60 seconds waiting to read random data from the kernel may have been printed at nearly the exact same time as the proceeding log message time="2022-10-09T21:33:15.541012931Z" level=info msg="Starting up"... or, as suggested by those two specific messages themself, the first line may have been printed many seconds before the second line. Since the first lines' real datetime of occurence cannot be known, it is just dropped, as it's preferable not to mislead the user.
So the first few lines of text was deliberately dropped.

However, maybe a special case should be made here. Upon deciding upon a datetimestamp regex for the file, any preceding lines should be included as part of that first log message.
Though, changing that presumption, the first line of text always has the datetimestamp substring, would complicate several major components including SyslogProcessor and printers.rs.

@jtmoon79 jtmoon79 added bug Something isn't working enhancement New feature or request P2 less important labels Apr 20, 2024
@jtmoon79 jtmoon79 added difficult A difficult problem; a major coding effort or difficult algorithm to perfect and removed bug Something isn't working labels Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
difficult A difficult problem; a major coding effort or difficult algorithm to perfect enhancement New feature or request P2 less important
Projects
None yet
Development

No branches or pull requests

1 participant