Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support parsing .json files with JSON query datetimestamp keying #309

Open
jtmoon79 opened this issue Jun 27, 2024 · 0 comments
Open

Support parsing .json files with JSON query datetimestamp keying #309

jtmoon79 opened this issue Jun 27, 2024 · 0 comments
Labels
difficult A difficult problem; a major coding effort or difficult algorithm to perfect enhancement New feature or request file parser

Comments

@jtmoon79
Copy link
Owner

jtmoon79 commented Jun 27, 2024

Summary

s4 cannot meaningfully parse this JSON

{[
 "logs": {
   "1": {
       "data": "FOO",
       "datetime": "20240101T000111-0700"
    }
    "2": {
       "data": "BAR",
       "datetime": "20240202T000222-0700"
    }
  },
  "attributes": {
    "yes": 1,
    "no: 0
  }
]}

It will match at the lines with "datetime" key and value only by way of line-oriented matching of the ISO 8601 datetime substring.

The first log message matched will be

       "datetime": "20240101T000111-0700"
    }
    "2": {
       "data": "BAR",

The second log message matched will be

       "datetime": "20240202T000222-0700"
    }
  },
  "attributes": {
    "yes": 1,
    "no: 0
  }

This is a very imprecise match and becoming non-sensical.

Suggested behavior

Add a CLI option that specifies a JSON pathing string (think jq filters). For .json files, this JSON query would be run to find the datetime.
But also, how to specify what element is the log message? For the example above, how to specify that each entry under logs is a log message?

TODO: look into jq filters and JSON querying

Additionally, the JSON querying may need to be different for different files. At teh same time, if the user is passing three JSON files that have the same query, don't force them to write that same JSON query three times on the command-line. How to have a kind of context-sensitive association from the CLI?

TODO: search clap project for an answer

@jtmoon79 jtmoon79 added enhancement New feature or request difficult A difficult problem; a major coding effort or difficult algorithm to perfect file parser labels Jun 27, 2024
@jtmoon79 jtmoon79 changed the title Support parsing .json files with queried datetimestamp keys Support parsing .json files with JSON query datetimestamp keying Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
difficult A difficult problem; a major coding effort or difficult algorithm to perfect enhancement New feature or request file parser
Projects
None yet
Development

No branches or pull requests

1 participant