Skip to content

Conference call notes 20230118

Jump to bottom
Kenneth Hoste edited this page Jan 18, 2023 · 2 revisions

(back to Conference calls)

Notes on the 214th EasyBuild conference call, Wednesday 18 January 2023 (16:00 UTC)

Attendees

List of attendees (12):

  • Sebastian Achilles (JSC, Germany)
  • Simon Branford (Univ. of Birmingham, UK)
  • Jasper Grimm (University of York, UK)
  • Alexander Grund (TU Dresden, Germany)
  • Kenneth Hoste (HPC-UGent)
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Kurt Lust (UAntwerpen & LUST)
  • Sebastien Moretti (SIB, Switzerland)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Bart Oldeman (Digital Research Alliance of Canada)
  • Jurij Pečar (EMBL, Germany)
  • Jörg Saßmannshausen (Imperial College London, UK)

Agenda

  • overview of recent developments + outlook to next EasyBuild release
  • update on porting of EasyBuild docs to MarkDown
  • Q&A

Recent developments

  • release timeline
    • latest release: EasyBuild v4.7.0 (9 Jan 2023)
      • includes foss/2022b + intel/2022b common toolchains
      • support for running on top of Python 2 is deprecated (prints a big fat warning)
      • distutils.version.LooseVersion is vendored in easybuild.tools.LooseVersion
      • enhancements/changes in experimental support for easystack files
      • gfbf as subtoolchain of foss + nvofbf toolchain
      • auto-enable use of oneAPI C/C++ compilers for intel-compilers >= 2022.2.0
      • JuliaPackage + JuliaBundle generic easyblocks
      • PythonPackage easyblock is now compatible with --sanity-check-only
      • framework/easyblocks unit tests are run with Python 3.11
      • support for 107 new software + tons of updated software versions
    • ETA next release (EasyBuild v4.7.1): mid Feb
  • recent changes
    • framework
      • bug fixes
        • add VERSION to easybuild.easyblocks namespace in test sandbox (PR #4171)
        • also apply filter to asyncprocess test subsuite (PR #4172)
        • don't use deprecated SafeConfigParser when running with Python 3.x (PR #4173)
        • filter out deprecation warnings for platform.dist in get_os_version and platform.linux_distribution in get_os_name (PR #4175)
        • update CI workflows to run 'apt-get update' if installation of packages via 'apt-get install' failed, likely due to stale apt cache (PR #4176)
      • enhancements
        • add support for using --output-format=md (MarkDown) (PR #4117, PR #4168)
        • add Ubuntu friendly package naming scheme EasyBuildDebFriendlyPNS (PR #4115)
        • accept single source as dict and single checksum in check_checksums_for (PR #4180)
      • changes
        • only give read permissions in GitHub Actions workflows (PR #4182)
        • remove Travis CI configuration file (PR #4174)
      • docs
        • ...
    • easyblocks
      • bug fixes
        • do not use -g77 option when installing NVHPC 22.9+ (PR #2819)
      • enhancements
        • ...
      • updates
        • update MesonNinja easyblock for Meson >=0.64.0 (PR #2861)
      • changes
        • only give read permissions in GitHub Actions workflows (PR #2863)
      • new software
        • ...
    • easyconfigs
      • ~75 easyconfig PRs were merged since last conf call
      • bug fixes
        • fix postinstallcmds and clarify download instructions for netMHC-4.0a (PR #9739)
        • fix postinstallcmds and clarify download instructions for netMHCpan-4.0a (PR #9740)
        • fix OpenBLAS 0.3.15 patch to correctly set the CPU core type for Tiger Lake (PR #15845)
        • add alternative checksum for UCX 1.13.1 after source tarball was changed in-place (without actual code changes) (PR #17077)
        • explictely download qe-gipaw source tarball from qe-gipaw GitHub repository in QuantumESPRESSO 7.1 easyconfigs (PR #17129)
        • add patch for GCCcore 11.1.0 + 11.2.0 to fix AVX2 bug (PR #17135)
          • very difficult to check whether software installations built with GCCcore/11.2.0 are affected by this
          • reinstalling software built on top rebuilt compiler may be neccessary, although it only affects software using a particular instruction - could affect BLAS, OpenMPI, etc.
        • fix source URL for p7zip v17.x, moved to p7zip-project repo (#17144)
      • enhancements
      • (noteworthy) new software
        • see also overview at https://github.com/easybuilders/easybuild-easyconfigs/issues?q=is%3Amerged+label%3Anew
        • UCX-ROCm (PR #17033)
        • nvofbf/2022.07 toolchain, incl. NVHPC v22.7, FFTW v3.3.10, FlexiBLAS 3.0.4, OpenBLAS v0.3.20, ... (PR #16724)
          • use case is mostly VASP
          • still some issues with BLIS (which doesn't seem to support getting built with NVHPC currently), not included as a FlexiBLAS backend in this toolchain, can be fixed if there's a need
          • can't use gfortran in this toolchain, because FlexiBLAS gets built with Intel ABI (calling conventions)
      • noteworthy software updates
      • changes
        • ...
  • work in progress
    • framework
      • reported bugs / bug fixes
        • Python packages are being installed to local/lib/python*/dist-packages due to tweaks to sysconfig.py in OS (issue #4128)
        • improve error when checksum dict has no entry for a file (PR #4150)
        • fix to_checksums with None values in dicts and recursion (PR #4159)
        • fix the checksum type check (PR #4164)
        • fix use of locate in check_os_dependency (PR #4166)
        • use better test assertions by replacing use of assertFalse/assertTrue (PR #4170)
      • enhancements
        • allow to directly import EasyConfig constants from the module (PR #4144)
        • enhance documentation of checksums easyconfig parameter (docs PR #853)
      • changes
        • ...
    • easyblocks
    • easyconfigs

Update on porting of EasyBuild docs to MarkDown

Q&A

  • Mikael: iomkl vs iofbf (+ iifbf?)
    • MKL or FlexiBLAS for BLAS/LAPACK
    • iofbf would imply losing compatibility with intel w.r.t. subtoolchains (compiler+BLAS/LAPACK only)
    • build systems need to be aware of FlexiBLAS
    • ComputeCanada has been using iofbf (but toolchains are not exposed to end users)
      • FlexiBLAS defaults to BLIS on AMD systems, MKL on Intel systems
      • in-house framework patch to link to -lflexiblas-intel when using Intel compilers
    • should we consider using FlexiBLAS also in intel toolchain, with MKL as default backend?
      • could be problematic for software that uses symbols from other parts of MKL (outside of BLAS/LAPACK)
    • another aspect is 64-bit integer support
      • FlexiBLAS does have support for this (but we may not be using it currently)
    • would support for listing multiple toolchains in a single easyconfig help?
      • we would still need to test with all these toolchains
      • we currently don't run tests with using different FlexiBLAS backend (like BLIS)
  • Mikael: anyone have experience/in-depth knowledge with EGL?
    • JSC has some experts there, we should check if they're open to give an EasyBuild Tech Talk on this topic?
    • VirtuaGL implemented support for EGL, big help for rendering on headless nodes
    • supports leveraging GPUs for using rendering buffer without having a screen
    • hardware vs software rendering
    • see also Mikael's repo @ https://github.com/Micket/gears
  • Which CUDA to use for 2022b?
    • CUDA 12.x is the first one to support H100 (?)
    • some trouble emerged with UCC and CUDA 12.0.0
  • Mikael: reluctant to port over CuPy patch to newer versions
Clone this wiki locally