Skip to content

Commit

Permalink
Merge branch 'main' into queryOptionButton-fix
Browse files Browse the repository at this point in the history
# Conflicts:
#	package-lock.json
#	src/contexts/StateContextProvider.tsx
  • Loading branch information
Henry8192 committed Dec 3, 2024
2 parents 698dd28 + 969ff35 commit f7e3698
Show file tree
Hide file tree
Showing 60 changed files with 5,708 additions and 1,556 deletions.
21 changes: 21 additions & 0 deletions .github/problem-matchers/stylelint.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"problemMatcher": [
{
"owner": "stylelint",
"pattern": [
{
"regexp": "^([^\\s].*)$",
"file": 1
},
{
"regexp": "^\\s*(\\d+):(\\d+)\\s+(\\S+)\\s+(.+?)\\s+(\\S+)$",
"line": 1,
"column": 2,
"message": 4,
"code": 5,
"loop": true
}
]
}
]
}
14 changes: 13 additions & 1 deletion .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ concurrency:
jobs:
lint-check:
runs-on: "ubuntu-latest"
env:
STYLELINT_MATCHER_OWNER: "stylelint"
STYLELINT_MATCHER_PATH: "./.github/problem-matchers/stylelint.json"
steps:
- uses: "actions/checkout@v4"
with:
Expand All @@ -29,4 +32,13 @@ jobs:
with:
node-version: 22
- run: "npm clean-install"
- run: "npm run lint:ci"
- name: "Set up Stylelint Problem Matcher"
run: "echo '::add-matcher::${{env.STYLELINT_MATCHER_PATH}}'"
- name: "Run `lint:check` script."
run: |-
# Disable ANSI color codes in Stylelint default "string" formatter
export NO_COLOR=1
npm run lint:check
- if: "always()"
name: "Remove Stylelint Problem Matcher"
run: "echo '::remove-matcher owner=${{env.STYLELINT_MATCHER_OWNER}}::'"
33 changes: 33 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: "test"

on:
pull_request:
types: ["opened", "reopened", "synchronize"]
push:
schedule:
# Run daily at 00:15 UTC (the 15 is to avoid periods of high load)
- cron: "15 0 * * *"
workflow_dispatch:

permissions: {}

concurrency:
group: "${{github.workflow}}-${{github.ref}}"

# Cancel in-progress jobs for efficiency
cancel-in-progress: true

jobs:
test:
runs-on: "ubuntu-latest"
steps:
- uses: "actions/checkout@v4"
with:
submodules: "recursive"
- uses: "actions/setup-node@v4"
with:
node-version: "22"
cache: "npm"
cache-dependency-path: "./package-lock.json"
- run: "npm clean-install"
- run: "npm run test"
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
/node_modules

# Build
/.task
/build
/dist

# For bundle size analysis
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "tools/yscope-dev-utils"]
path = tools/yscope-dev-utils
url = https://github.com/y-scope/yscope-dev-utils.git
83 changes: 26 additions & 57 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,83 +4,52 @@
![Open feature requests][feature-requests-shield]
![CLP on Zulip][zulip-shield]

`yscope-log-viewer` is a tool that can be used to view log files. It currently
supports [CLP][clp-repo]'s compressed log files (IR streams) and JSON log files.
The viewer can be used to navigate the log file, filter by log level, view the
logs with syntax highlighting, and generate direct links to specific log events.
The YScope Log Viewer is a tool that can be used to view log files. It currently supports
[CLP][clp-repo]'s compressed log files (IR streams) and JSON log files. The viewer can be used to
navigate the log file, filter by log level, view the logs with syntax highlighting, and generate
direct links to specific log events.

See the [online demo](#online-demo) section to try out the log viewer on a
sample log file. To set up a local server, follow the
[build guide](docs/dev-guide/building.md).
See the [demo](#demo) section to try the log viewer on a sample log file. Or you can generate your
own compressed log files using one of our [libraries][docs-site/generating-ir-stream-logs].

See the [features in development](#features-in-development) section for upcoming
features.
Want to report a bug or request a feature? Check out the [feedback](#providing-feedback) section.
A list of [features in development][docs-site/features-in-development] is available on our
[docs site][docs-site].

# Online Demo
# Demo

A demo of the viewer available [here][online-demo].

* A demo of the log viewer can be found [here][online-demo].
* The demo loads a Hadoop YARN log file from the [hive-24hrs] log dataset.
* More info on the dataset and other datasets can be found [here][datasets].
* To open an IR stream, drag and drop it onto the log viewer or use the open
file dialog.

# Generating IR Stream Logs

IR stream log files can currently be generated using these libraries:

* [Log4j Logging Library][log4j1-appenders]
* [Logback Logging Library][logback-appenders]
* [Python Logging Library][clp-loglib-py]
* Golang Logging Library (in development)

# How does it work?
* To open an IR stream, drag and drop it onto the log viewer or use the open file dialog.

The log viewer is written using the ReactJS framework and uses the open source
[monaco-editor] and [clp-ffi-js].
# Docs

The viewer spawns a worker to assist with computationally intensive tasks such
as:
The log viewer's docs are available [here][docs-site].

* Deserializing the file and creating an index of logs events.
* Paginating the indexed logs.
* Decoding the deserialized log events into plain text.
# Providing feedback

Tasks are passed to the worker as needed and changes are rendered in the UI.

# Providing Feedback

You can use GitHub issues to [report a bug][report-bug] or
[request a feature][request-feature].
You can use GitHub issues to [report a bug][bug-report] or [request a feature][feature-req].

Join us on [Zulip][zulip] to chat with developers and other community members.

# Contributing

See the docs in our [developer guide](docs/dev-guide).

# Features in Development

* Pretty printing to enhance the readability of structured data in the logs.
* A dashboard to visualize the distribution of log types and log levels.
* Support for plain text, archived log files, and other requested formats.
* Searching within a file, multiple files, or within provided time ranges.
* Infinite scrolling instead of pagination.
* Log correlation with sync by timestamp across multiple editors.
* Automatic conversion of text log files to IR stream format in the browser.
* Deployment of components via NPM.
See the docs in our [developer guide][docs-site/dev-guide].

[bug-report]: https://github.com/y-scope/yscope-log-viewer/issues/new?labels=bug&template=bug-report.yml
[bugs-shield]: https://img.shields.io/github/issues/y-scope/yscope-log-viewer/bug?label=bugs
[clp-ffi-js]: https://github.com/y-scope/clp-ffi-js
[clp-loglib-py]: https://github.com/y-scope/clp-loglib-py
[clp-repo]: https://github.com/y-scope/clp
[datasets]: https://docs.yscope.com/clp/main/user-guide/resources-datasets
[docs-site]: https://docs.yscope.com/yscope-log-viewer/main/
[docs-site/building]: https://docs.yscope.com/yscope-log-viewer/main/dev-guide/building-getting-started
[docs-site/dev-guide]: https://docs.yscope.com/yscope-log-viewer/main/dev-guide/index
[docs-site/features-in-development]: https://docs.yscope.com/yscope-log-viewer/main/index#features-in-development
[docs-site/generating-ir-stream-logs]: https://docs.yscope.com/yscope-log-viewer/main/index#generating-ir-stream-logs
[feature-req]: https://github.com/y-scope/yscope-log-viewer/issues/new?labels=enhancement&template=feature-request.yml
[feature-requests-shield]: https://img.shields.io/github/issues/y-scope/yscope-log-viewer/enhancement?label=feature-requests
[hive-24hrs]: https://zenodo.org/record/7094921#.Y5JbH33MKHs
[log4j1-appenders]: https://github.com/y-scope/log4j1-appenders
[logback-appenders]: https://github.com/y-scope/logback-appenders
[monaco-editor]: https://microsoft.github.io/monaco-editor/
[hive-24hrs]: https://zenodo.org/records/7094921#.Y5JbH33MKHs
[online-demo]: https://y-scope.github.io/yscope-log-viewer/?filePath=https://yscope.s3.us-east-2.amazonaws.com/sample-logs/yarn-ubuntu-resourcemanager-ip-172-31-17-135.log.1.clp.zst
[report-bug]: https://github.com/y-scope/yscope-log-viewer/issues/new?labels=bug&template=bug-report.yml
[request-feature]: https://github.com/y-scope/yscope-log-viewer/issues/new?labels=enhancement&template=feature-request.yml
[zulip]: https://yscope-clp.zulipchat.com/
[zulip-shield]: https://img.shields.io/badge/zulip-yscope--clp%20chat-1888FA?logo=zulip
46 changes: 46 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Docs

This directory contains the files necessary to generate a Sphinx-based documentation website for
this project:

* `conf` - Configuration files
* `src` - The actual docs

## Requirements

* [Git LFS][git-lfs]
* We use Git LFS to store the images in the docs; this is to avoid significant increases in
the size of repo as we add and update images.
* [Node.js] >= 16 to be able to [view the output](#viewing-the-output)
* Python 3.10 or later
* [Task] >= 3.38.0

## Build commands

* Build the site incrementally:

```shell
task docs:site
```

* The output of the build will be in `../build/docs/html`.

* Clean up the build:

```shell
task docs:clean
```

## Viewing the output

```shell
task docs:serve
```

The command above will install [http-server] and serve the built docs site; `http-server` will print
the address it binds to (usually http://localhost:8080).

[git-lfs]: https://git-lfs.com
[http-server]: https://www.npmjs.com/package/http-server
[Node.js]: https://nodejs.org/en/download/current
[Task]: https://taskfile.dev/
78 changes: 78 additions & 0 deletions docs/conf/conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# -- Project information -------------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information

project = "YScope Log Viewer"

# NOTE: We don't include a period after "Inc" since the theme adds one already.
copyright = "2022-2024 YScope Inc"

# -- General configuration -----------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

extensions = [
"myst_parser",
"sphinx_copybutton",
"sphinx_design",
"sphinx.ext.autodoc",
"sphinx.ext.viewcode",
"sphinxcontrib.mermaid",
]

# -- MyST extensions -----------------------------------------------------------
# https://myst-parser.readthedocs.io/en/stable/syntax/optional.html
myst_enable_extensions = [
"attrs_block",
"colon_fence",
]

myst_heading_anchors = 4

# -- Sphinx autodoc options ----------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#configuration

autoclass_content = "class"
autodoc_class_signature = "separated"
autodoc_typehints = "description"

# -- HTML output options -------------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output

html_favicon = "https://docs.yscope.com/_static/favicon.ico"
html_title = project
html_show_copyright = True

html_static_path = ["../src/_static"]

html_theme = "pydata_sphinx_theme"

# -- Theme options -------------------------------------------------------------
# https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/layout.html

html_theme_options = {
"footer_start": ["copyright"],
"footer_center": [],
"footer_end": ["theme-version"],
"navbar_start": ["navbar-logo"],
"navbar_end": ["navbar-icon-links", "theme-switcher"],
"primary_sidebar_end": [],
"secondary_sidebar_items": ["page-toc", "edit-this-page"],
"show_prev_next": False,
"use_edit_page_button": True,
}

# -- Theme source buttons ------------------------------------------------------
# https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/source-buttons.html

html_context = {
"github_user": "y-scope",
"github_repo": "yscope-log-viewer",
"github_version": "main",
"doc_path": "docs/src",
}

# -- Theme custom CSS and JS ---------------------------------------------------
# https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/static_assets.html


def setup(app):
app.add_css_file("custom.css")
30 changes: 0 additions & 30 deletions docs/dev-guide/building.md

This file was deleted.

6 changes: 0 additions & 6 deletions docs/dev-guide/state-and-stateRef.md

This file was deleted.

16 changes: 0 additions & 16 deletions docs/dev-guide/validation.md

This file was deleted.

Loading

0 comments on commit f7e3698

Please sign in to comment.