EESSI SURF visualisation 2023 10 11

Meeting with SURF visualization team w.r.t. EESSI (2023-10-11)

• attendees

  • Caspar van Leeuwen (SURF, EESSI + MultiXscale)
  • Casper van Leeuwen (SURF)
  • Xin An (SURF, MultiXscale)
  • Kenneth Hoste (HPC-UGent, EESSI + MultiXscale)
  • Sam Moors (VUB)

• overview of software available in EESSI

  • table overview
    • columns: CPU targets (currently 8, could grow a bit, but not 100)
    • rows: software names
  • page per software with software versions
  • visual software "status"
    • green tick: software is available, working fine
    • orange dot: software is installed, but known problems
    • red cross: software is missing
  • link to known issues with more info on why that's missing
    • see
  • could do a similar setup to software overview in HPC-UGent docs
    • see https://docs.hpc.ugent.be/available_software
    • generated with Javascript https://github.com/hpcugent/vsc_user_docs/tree/main/mkdocs/docs/HPC/only/gent/available_software
    • script used to generate JSON input files in https://github.com/hpcugent/vsc_user_docs/tree/main/scripts/available_software

• result of EESSI test suite

  • is software included in EESSI still working/performing as expected

• actions

  • EESSI can look into a script to generate JSON file(s) for software available in EESSI

• plan is to add software overview in EESSI docs (https://eessi.io/docs/software)

  • landing page:
    • only software names as rows
    • CPU targets as column names
    • dropdown list or links with EESSI versions (with a default)
    • search box to search across EESSI versions
    • expand/indent with software versions when you click a software name (like in GitHub tree)
      • like here on the left
  • page per software
    • module names as rows (for that software)
    • CPU targets as columns
    • info specific to the software
    • link to external dashboard with test/performance result for a particular software installation
      • ideally in VM hosted in AWS/Azure by EESSI

• current SURF dashboard

  • see https://monitor.hpcv-performanc.src.surf-hosted.nl
  • MariaDB database that hosts data extracted from ReFrame performance logs
  • TypeScript for dashboard, Python API to get database from DB
  • looks very nice, some tweaks could be useful for EESSI
  • additional filters would be useful, like tags (1_node, etc.), test hash, ...
  • for EESSI, we're mostly interested in performance over time of a particular test on a particular system