Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bench | locli: new DB persistence backend for analysis data #6054

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

mgmeier
Copy link
Contributor

@mgmeier mgmeier commented Dec 11, 2024

Description

This PR adds a new database persistence backend to the benchmark analysis tool locli.
Persisting log objects in a serverless SQLite DB has proven several advantages over the current file based solution (FS). Considering a perf cluster run with abundant raw log data:

  • On-disk space: 5.5GiB (DB) - 66GiB (FS)
  • Standard analysis wall time: 26min (DB) - 41min (FS)
  • Faster conversion of raw log data to analysis input (not measured, only observed)

Furthermore, the DB backend opens up a way to carry out custom quick queries into the analysis input - either on the application level or in SQL. A suitable abstraction for those will be a future endeavor; this PR adds a new testbed application locli-quick for development (which, for that reason, needs not be a workbench shell input as of now).

Last not least, default RTS GC settings for locli have been revisited. This PR changes the default such that on the perf cluster deployer, a standard analysis (almost) fits into 64GiB, and as a result there's (almost) no swapping going on. This is translates into a significant speed improvement, regardless of which persistence backend is used.

The PR implements all changes listed in locli's changelog, bumping it to a new major version 2.0:

  • New database (DB) persistence backend for log objects using serverless SQLite DBs
  • Refactor current file persistence backend into its own module
  • New CLI commands prepare-db and unlog-db to create and read from DB persistence backend respectively
  • New sum type LogObjectSource to represent input from different backends (file or DB)
  • Tweak GC to mitigate high RAM requirements (for perf cluster analyses only)
  • New executable locli-quick which aims to be a development testbed for (upcoming) DB-backed quick queries

It also integrates the new locli feature into workbench such that:

  • a new envvar $WB_LOCLI_DB is introduced, defaulting to 0
  • it can be set from an outer env, from within the workbench shell, or by the flag --locli-db
  • setting it to 1 results in using the DB for both preparation and analysis steps

This means the changes of this PR are currently "opt-in". All existing functionality is retained, unmodified, and as such, this PR is non-breaking wrt. to existing workflows.

Checklist

  • Commit sequence broadly makes sense and commits have useful messages
  • New tests are added if needed and existing tests are updated. These may include:
    • golden tests
    • property tests
    • roundtrip tests
    • integration tests
      See Runnings tests for more details
  • Any changes are noted in the CHANGELOG.md for affected package
  • The version bounds in .cabal files are updated
  • CI passes. See note on CI. The following CI checks are required:
    • Code is linted with hlint. See .github/workflows/check-hlint.yml to get the hlint version
    • Code is formatted with stylish-haskell. See .github/workflows/stylish-haskell.yml to get the stylish-haskell version
    • Code builds on Linux, MacOS and Windows for ghc-8.10.7 and ghc-9.2.7
  • Self-reviewed the diff

@mgmeier mgmeier requested review from a team as code owners December 11, 2024 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant