Skip to content

Conference call notes 20201209

Jump to bottom
Kenneth Hoste edited this page Dec 9, 2020 · 5 revisions

(back to Conference calls)

Notes on the 162nd EasyBuild conference call, Wednesday December 9th 2020 (16:00 UTC - 17:00 CET)

Attendees

Alphabetical list of attendees (13):

  • Sebatian Achilles (Jülich Supercomputing Centre, Germany)
  • Simon Branford (University of Birmingham, UK)
  • Alex Domingo (ULB, Belgium)
  • Fotis Georgatos (SDSC, Switzerland)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Robert Mijakovic (LuxProvide)
  • Pavlin Mitev (Uppsala Univ., Sweden)
  • Alan O'Cais (Jülich Supercomputing Centre, Germany)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Bart Oldeman (ComputeCanada)
  • Victor Rusu (CSCS, Switzerland)
  • Lars Viklund (Umeå University, Sweden)

Agenda

  • update on recent developments + upcoming EasyBuild release (v4.3.2)
  • considering switch to BLIS for foss common toolchains
  • agenda for (virtual) EasyBuild User Meeting
  • Q&A

Recent developments

  • recent changes
    • framework
    • easyblocks
      • bug fixes
        • don't set $PYTHONNOUSERSITE in Python extension filter, it breaks some installations (PR #2267)
        • fix setting of $RUNPARALLEL in HDF5 easyblock (PR #2250)
        • set $UCX_TLS to 'all' for impi installed on top of UCX (PR #2253)
        • enhance PyTorch easyblock to ensure it finds MKL (via $MKLROOT) (PR #2257)
        • fix hardcoded path in NVHPC easyblock to support multiple architectures (PR #2233)
      • enhancements
        • add support for building Clang with OpenMP offload support (PR #2229)
        • add support for withnvptx easyconfig parameter, to enable GPU offloading, in GCC easyblock (PR #2235)
        • allow wxPython to be installed as an extension (PR #2227)
            • follow-up to add (back) symbolic links for libwx_*so.0 files in wxPython installation (PR #2275)
        • also add 'bin' subdir to $PATH when installing a Python package (PR #2244)
      • new software
        • new custom easyblocks for code-server (PR #2255) and Metagenome-Atlas (PR #2219)
    • easyconfigs
      • bug fixes
        • patch to fix possible memory leak in OpenBLAS (PR #11745)
        • don't add include/node to $CPATH in nodejs easyconfigs + tweak V8 easyconfigs accordingly (PR #11845)
          • this broke the installation of R v4.0.3 (see PR #11663) on CentOS 7
          • follow-up in issue #11846 w.r.t. making nodejs use system OpenSSL
        • add mising gnuplot dependency for OpenFOAM (PR #11770 + PR #11800)
          • see also enhanced sanity check in OpenFOAM easyblock at PR #2256
      • enhancements
        • enable NVPTX offload support in GCCcore 10.2.0 easyconfig (PR #11720)
          • we'll probably also enable this for older GCCcore versions (up to 7.x)
      • new software
        • (nothing special)
      • software updates
        • (nothing special)
      • easyconfigs for 2020b generation
      • changes
        • renaming (for consistency/correctness)
  • to merge soon
    • framework (v4.4.0 milestone)
      • support additional features in easystack files (see issues #3468, #3512, #3513, #3516)
      • directories that don't contain any library files shouldn't be added to $LD_LIBRARY_PATH (issue #3504)
    • easyblocks (v4.4.0 milestone)
      • bug fixes
        • correctly determine path to active binutils in TensorFlow easyblock (PR #2218)
        • fix taking into account --sysroot when installing/using CMake
          • two options:
            • use toolchain file in CMakeMake (PR #2247)
            • patch CMake installation (PR #2248)
        • check scipy test results (PR #2241)
          • some scipy tests are failing, have to tweak accuracy tolerances?
      • enhancements
        • enhance OpenBLAS easyblock to make it aware of optarch (PR #1946)
        • run motorBike tutorial case for recent (community) OpenFOAM versions (PR #2201)
          • needs testing...
        • add support for statically linking Bazel (PR #2272)
        • add options to run unit tests to TensorFlow easyblock (PR #2263)
      • new software
    • easyconfigs (v4.4.0 milestone)
      • bug fixes
        • added missing space in configopts in ParaView 5.8.0 easyconfigs using 2020a toolchain (PR #10989)
          • (cfr. discussion last conf call)
      • enhancements
      • new software
        • gobff toolchains (PR #11761)
          • exceptions need to be added to tests
      • software updates

BLIS for foss common toolchains?

  • follow-up on discussion with Sebastian Achilles (JSC)
  • see notes
  • PR #11761 for gobff toolchains
  • some results on Intel Skylake (Sebastian)
    • BLIS-based toolchains are less performant compared to MKL, but better than OpenBLAS
    • mainly BLAS-3 function are faster in BLIS compared to MKL (certainly on AMD), but also BLAS-1
    • some LAPACK functions are not parallelized in libFLAME (but are in MKL)

Easyconfigs merge sprint

  • Fri Dec 11th 2020!

Agenda for (virtual) EasyBuild User Meeting

  • see https://github.com/easybuilders/easybuild/wiki/6th-EasyBuild-User-Meeting
  • TO PLAN:
    • talk on EESSI project
    • talk on Gentoo Prefix
    • talk on PRACE Best Practice Guide
    • talk on OpenHPC?
    • talk on Cluster-in-the-Cloud?
    • Singularity demo or workshop by Jörg
      • talk to introduce Singularity (Mon)
      • multiple short workshops
    • site presentations (~15min + Q&A)
    • JSC
    • CSCS?
    • AstraZeneca (to be confirmed)
    • other?

Q&A

  • Mikael: diamond-DAG for 2021a toolchains?
    • make a foss/2021.00 (same GCCcore as in foss/2020b)
    • set up a separate meeting on this?
  • Alex: interest in toolchain based on AMD compiler?
    • Victor: already AMD-compiler-based toolchain on Cray
    • nice time to implement framework feature to "map" easyconfigs from toolchain A to toolchain B
      • eb R-4.0.0-foss-2020a.eb --map-to-toolchain intel,2020a --robot
      • see --try-toolchain --robot + --try-update-deps?
      • or as alternative: support 'toolchains' in easyconfigs (list of working toolchains)
        • this has bigger impact w.r.t. robot search, CLI, etc.
  • Victor: minimal set of "key" easyconfigs tied to EasyBuild features?
    • like FPM, etc.
    • better testing for specific EasyBuild features like this?
    • all boils down to manpower and focus of people actively contributing to EasyBuild on features they use...
  • Mikael: can we ask additional maintainers to get involved?
    • cfr. POWER, testing by Simon at Univ. of Birmingham
    • we should clarify the role of maintainers better (maintainer vs people with merge rights)
  • Robert: how can we (LuxProvide) help out with support for support for AMD-compiler-based toolchains?
    • tasks would be:
      • add support for installing AMD compiler (AOCC)
      • add toolchain definition to EasyBuild framework (see Clang-based toolchains for example)
      • add easyconfigs for AOCC-based toolchain (cfr. recent gobff PR)
    • also help out with regression testing for EasyBuild, testing contribution (PRs), performance evaluation, etc.
Clone this wiki locally