Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Add user docs for opening log files, level filtering, and formatting structured logs. #132

Merged
merged 26 commits into from
Nov 28, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
bb5617a
added basic user docs
davemarco Nov 27, 2024
28067b6
Refactor user-guide/index.md
kirkrodrigues Nov 28, 2024
637b520
Refactor user-guide/quick-start-overview.md
kirkrodrigues Nov 28, 2024
a3d053b
Refactor user-guide/format-string-overview.md
kirkrodrigues Nov 28, 2024
941b558
Refactor user-guide/format-string-syntax.md
kirkrodrigues Nov 28, 2024
60aade6
Refactor user-guide/format-string-formatters.md
kirkrodrigues Nov 28, 2024
8d7d10a
Refactor user-guide/log-level-filter.md
kirkrodrigues Nov 28, 2024
59f3e22
kirk review
davemarco Nov 28, 2024
42e1f43
fix broken link
davemarco Nov 28, 2024
fd19fa2
fix more broken things..
davemarco Nov 28, 2024
c2e92cf
more
davemarco Nov 28, 2024
fbd51cd
Fix typo.
kirkrodrigues Nov 28, 2024
e18af3c
Add images to git LFS.
kirkrodrigues Nov 28, 2024
68ab84e
Fix typo.
kirkrodrigues Nov 28, 2024
6ca3093
Add some commas for clarity.
kirkrodrigues Nov 28, 2024
163d2c6
Revert "Add images to git LFS."
kirkrodrigues Nov 28, 2024
2c9d621
Re-add images as normal files.
kirkrodrigues Nov 28, 2024
2e774dc
Retry adding images as normal files.
kirkrodrigues Nov 28, 2024
a0742a6
Retry adding images as normal files.
kirkrodrigues Nov 28, 2024
893820e
Remove images.
kirkrodrigues Nov 28, 2024
04442ef
Update docs/src/user-guide/log-level-filtering.md
davemarco Nov 28, 2024
2efe385
Update docs/src/user-guide/log-level-filtering.md
davemarco Nov 28, 2024
e60e376
add lfs
davemarco Nov 28, 2024
32605d2
Update docs/src/user-guide/format-struct-logs-syntax.md
davemarco Nov 28, 2024
7d56440
Update docs/src/user-guide/format-struct-logs-overview.md
davemarco Nov 28, 2024
6c0359b
Add nested field to structured log formatting example; Replace curly …
kirkrodrigues Nov 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/src/attachments/individual-filter.png
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Can we move the images into the directory where they're used (i.e., user-guide)?
  • Can we enable LFS for docs/src like we do in CLP?

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/src/attachments/severity-filter.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
33 changes: 33 additions & 0 deletions docs/src/user-guide/format-string-formatters.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the format-struct-logs- syntax instead (to match the sidebar caption)?

Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Formatters

Formatters allow you to transform how field values are displayed. See below for a list of
formatters.

## Timestamp Formatter

Converts Unix timestamps to human-readable date-time strings.

**Usage:**
`{field:timestamp[:options]}`

**Options:**
- Accepts a format string specifically for the date.
- Uses [Day.js format tokens](https://day.js.org/docs/en/display/format).
- Date string must escape colons with a backslash, e.g., `HH\:mm\:ss`.
- If no format is specified, the default format is ISO 8601.

**Examples:**
- `{ts:timestamp}` → `2024-11-27T10:30:00Z`
- `{ts:timestamp:YYYY-MM-DD}` → `2024-11-27`

---

## Round Formatter

Rounds a numeric value to the nearest integer.

**Usage:**
`{field:round}`

**Example:**
- `{value:round}` → `5.7` becomes `6`
35 changes: 35 additions & 0 deletions docs/src/user-guide/format-string-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Overview

The log viewer can accept a format string to transform JSON logs into human-readable text.
You can select which fields to display and how they should be formatted. The format string
input can be found in the settings dialog.

For example, for a JSON log with many fields:

```json
{
"ts": 1732733160216,
"level": "ERROR",
"message": "Failed to process payment transaction",
"trace_id": "abc123def456",
"span_id": "span_789xyz",
"service": "payment-service",
"environment": "production",
}
```

You might want to show only:
- timestamp
- log level
- message

You can achieve this with the following format string `{ts:timestamp} {level} {message}`.
`{ts:timestamp}` formats the timestamp field (Unix epoch) into a ISO 8601 date format.
`{level}` and `{message}` display the field as is.

The log is displayed as:
```
2024-11-27T18:46:00Z ERROR Failed to process payment transaction.
```

For detailed information about the format string syntax, see [Format String Syntax](format-string-syntax).
64 changes: 64 additions & 0 deletions docs/src/user-guide/format-string-syntax.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Format String Syntax

- The format string is composed of arbitrary (“static”) text and field placeholders enclosed in
braces (`{` and `}`).
- Each brace or backslash in the static text must be escaped by a backslash.
- Each field placeholder has the following form, consisting of three components (contained in
braces):

```
{<field-name>[:<formatter-name>[:<formatter-options>]]}
```

- `field-name` (required) defines the key of the field whose value should replace the
placeholder.
- Nested fields can be specified using periods (`.`) to denote hierarchy. E.g., the field
`{"a:" {"b": 0}}` may be denoted by `a.b`.
- Field names can contain any character except that each period, dollar sign, brace (left
or right), colon, or backslash must be escaped with a backslash.
- `formatter-name` (optional, unless `formatter-options` is specified) is the name of the
formatter to apply to the value before inserting it into the string.
- Formatter names can contain any character except a space (’ ‘) while each brace (left or
right), colon, or backslash must be escaped with a backslash.
- `formatter-options` (optional) defines any options for the formatter denoted by
`formatter-name`.
- Formatter options can contain any character except each brace (left or right), colon, or
backslash must be escaped with a backslash.
- Finally, every format string contains an implicit trailing newline so that each formatted log
event ends with a newline.

For example, consider the following log event:

```
{
"@timestamp": 1427153388942,
"level": "INFO",
"thread": 0,
"latency_secs": 56.4,
"an.odd.key{name}": "org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
hadoop-metrics2.properties"
}
```

We can format this using the following YScope format string:

```
{@timestamp:timestamp:YYYY-MM-DD HH\:MM\:ss.SSS} {level} \{{thread}\} latency={latency_secs:round}
{an\.odd\.key\{name\}}
```

- In the first placeholder, we have the field name `ts`, a formatter called `timestamp`, and the
formatter’s options which are a date format string.
- The second and third placeholders simply stringify the values of the given fields.
- The fourth placeholder uses the `round` formatter to round the field’s value; this placeholder
doesn’t specify any formatter options, so the defaults will be used.
- The fifth placeholder is for a field whose name contains characters that require escaping.

The formatted string will be:

```
2015-03-23 19:29:48.942 INFO {0} latency=56 org.apache.hadoop.metrics2.impl.MetricsConfig: loaded
properties from hadoop-metrics2.properties
```

For a list of currently supported formatters, see [Formatters](format-string-formatters).
49 changes: 48 additions & 1 deletion docs/src/user-guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,51 @@
This guide documents how to use and operate the YScope Log Viewer. Use the left sidebar (if it's
hidden, click the {fas}`bars` icon) to navigate to specific docs.

More details will be added soon.
::::{grid} 1 1 2 2
:gutter: 2

:::{grid-item-card}
:link: quick-start-overview
Quick start
^^^
A guide to open files in the log viewer.
:::

:::{grid-item-card}
:link: format-string-overview
Format String
^^^
Docs for format strings.
:::

:::{grid-item-card}
:link: log-level-filter
Log Level Filter
^^^
Docs for filtering by log level.
:::
::::

:::{toctree}
:hidden:
:caption: Quick start

quick-start-overview
:::

:::{toctree}
:hidden:
:caption: Format String
:glob:

format-string-overview
format-string-syntax
format-string-formatters
:::

:::{toctree}
:hidden:
:caption: Log Level Filter

log-level-filter
:::
15 changes: 15 additions & 0 deletions docs/src/user-guide/log-level-filter.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename to log-level-filtering (to match the sidebar caption)?

Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Overview

The log viewer offers two ways to filter log levels.

## Severity Filtering
Filter logs at or above a selected level. Click the level name (e.g., INFO) to show all logs at
or above that severity. For example, selecting INFO shows INFO, WARN, ERROR, and FATAL logs.

![Severity filtering](../attachments/severity-filter.png)

## Individual Filtering
Select specific log levels to show or hide. Use the checkboxes next to each level name to toggle
visibility.

![Individual level filtering](../attachments/individual-filter.png)
11 changes: 11 additions & 0 deletions docs/src/user-guide/quick-start-overview.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we drop -overview?

Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Overview

This guide shows you how to open local and remote log files.

## Local Files
To open a local file, use the folder icon (📁) in the top-left corner. This will open a file
browser where you can navigate to and select your log file.

## Remote Files
To open a remote file, append `/?filePath=<FILE_URL>` to the current URL. Just replace
`<FILE_URL>` with the URL of your log file.