Skip to content

Conference call notes 20240828

Jump to bottom
Kenneth Hoste edited this page Sep 10, 2024 · 2 revisions

(back to Conference calls)

Notes on the 253rd EasyBuild conference call, Wednesday 28 Aug 2024 (15:00 UTC)

Attendees

List of attendees (16):

  • Simon Branford (Univ. of Birmingham, UK)
  • Em Dragowsky (Case Western Reserve University, Ohio, US)
  • Alex Domingo (Vrije Universiteit Brussel, Belgium)
  • Davide Grassano (CECAM, Switzerland)
  • Alexander Grund (ZIH, Dresden, Germany)
  • Leonardo Honfi Camilo (Univ. of Wageningen, Netherlands)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Chia-Jung Hsu (Chalmers University of Technology, Sweden)
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Georgios Kafanas (University of Luxembourg)
  • Sam Moors (Vrije Universiteit Brussel, Belgium)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Simon Pinches (ITER)
  • Alex Salois (Montana State University, US)
  • Louwrens Van Dellen (ITER)
  • Cintia Willemyns (Vrije Universiteit Brussel, Belgium)

Agenda

  • overview of recent developments
  • outlook to EasyBuild 5.0 release
  • 2024a/2024b update of common toolchains
  • Q&A

Recent developments

  • latest EasyBuild release: 4.9.2 (12 June 2024)
  • next EasyBuild release:
    • likely EasyBuild 4.9.3 quite soon (days/weeks, as EasyBuild 5.0.0 is still weeks/months away at best...)
      • status:
        • framework: no more changes planned
        • easyblocks: (TBD)
        • easyconfigs: only bug fix PRs (and whatever gets merged before 4.9.x branch is updated with current develop to prepare for release)
  • EasyBuild v5.0.0
    • detailed notes on latest developments in https://hackmd.io/tZtz6vMmRfGL6N9Rf__OEA
    • to test development version of EasyBuild 5.0: 
      # set up Python virtual environment, and jump into it
python3 -m venv eb5
source eb5/bin/activate

# install EasyBuild 5.0 development version into it
pip install https://github.com/easybuilders/easybuild-framework/archive/5.0.x.tar.gz
pip install https://github.com/easybuilders/easybuild-easyblocks/archive/5.0.x.tar.gz
pip install https://github.com/easybuilders/easybuild-easyconfigs/archive/5.0.x.tar.gz

# go!
eb --version

Merged PRs

  • docs (merged PRs)

    • ...

  • framework (merged PRs)

    • bug fixes
      • fix help text for cuda_compute_capabilities template (PR #4589)
      • fix help message for --http-headers-fields-urlpat configuration option (PR #4594)
    • enhancements
      • ...
    • changes
      • ...
    • code cleanup
      • clean up code that was only there to support Python 2.6 + avoid syntax warnings when parsing py2vs3/py.p2 with Python 3.x (PR #3788)
    • EasyBuild 5.0 (5.0.x branch)
      • drop load storm safe guard for Environment Modules v4.2.4+ (PR #4373)
      • add check_group support for module files in Tcl syntax (PR #4418)
      • avoid suggesting that toolchainopts are supported for SYSTEM compiler, since they're not (PR #4585)
      • Use getenv modulefile command with EnvironmentModules >= 4.2.0 (PR #4614)
      • set usedforsecurity to False when calling hashlib.md5 with Python >= 3.9 (PR #4550)
      • let cmd.sh script produced by run_shell_cmd define $EB_CMD_OUT_FILE + $EB_CMD_ERR_FILE (PR #4611)
      • improve logging in helper function used by run_shell_cmd to answer questions for interactive shell commands (PR #4612)
      • Add module cache build support on EnvironmentModules (PR #4615)
      • fix run_shell_cmd to avoid that same question is answered multiple times (PR #4617)

  • easyblocks (merged PRs)

    • bug fixes
      • fix double initialization of Cargo by CargoPythonPackage by removing incorrect custom __init__ implementation + fix use of super() in PALM easyblock (since that doesn't work with Python 2.7) (PR #3406)
    • enhancements
      • enhance custom easyblock for LAMMPS: add sanity_check_test_inputs custom easyconfig parameter, add mapping for zen4, also support building of non-stable LAMMPS versions (PR #3336)
      • update custom easyblock for QuantumESPRESSO to be aware of MPI tests being disabled in EasyBuild configuration (PR #3412)
    • updates
      • update ORCA easyblock for version 6 (PR #3395)
      • update custom easyblock for psmpi to support the renaming of a couple of options in 5.10.0-1 (PR #3420)
    • changes
      • ...
    • new easyblocks
      • ...
    • code cleanup
      • ...
    • EasyBuild 5.0
      • Replace run_cmd with run_shell_cmd in custom easyblock for QuantumESPRESSO (PR #3411)
      • use run_shell_cmd in custom easyblock for WIEN2k (PR #3413)
      • replace run_cmd with run_shell_cmd in custom easyblock for MetalWalls (PR #3415)

  • easyconfigs (merged PRs)

    • ~75 easyconfig PRs were merged since last conf call
    • bug fixes/reports
    • enhancements
    • (noteworthy) new software
    • noteworthy software updates
    • changes
      • remove Cython dependency from Python-bundle-PyPI 2024.06 + standalone easyconfig for Cython 3.0.10 (PR #21233)
      • make pytest v7.4.2 independent of Python-bundle-PyPI (PR #21004)

Open (active) PRs

  • docs (open PRs + issues)

    • ...

  • framework (open PRs + issues)

    • bug fixes
      • ...
    • enhancements
      • add support for --sources-url to fetch sources from additional URLs (PR #4079)
      • Copy build log and artifacts to a permanent location after failures (WIP PR #4601)
    • changes
      • ...
    • EasyBuild 5.0
      • project board: https://github.com/orgs/easybuilders/projects/18/views/2
      • Enable module-depends-on by default (PR #4500)
        • only use depends_on when using Lmod as modules tool?
      • Make module-extensions true by default (PR #4501)
      • More granular exit codes (PR #4534)
      • prepend to $PYTHONPATH or $EBPYTHONPREFIXES in generated module files by automatically scanning for python site package directories #4539 (PR #4539
      • Use default value $XDG_CONFIG_DIRS from XDG basedir spec: /etc/xdg (instead of /etc) (PR #4591)
        • we should probably also still consider /etc/easybuild.d/*.cfg, or at least print a warning when configuration files exist there
      • convert template constant lists to dicts and export the constants by name (PR #4595)
        • use PYPI_SOURCE constant in generic PythonPackage easyblock + custom easyblock for Python (easyblocks PR #3410)
      • Run unit tests on an updated version of Modules 4 (PR #4415)

  • easyblocks (open PRs + issues)

    • bug fixes
      • update custom easyblock for PETSc to consider include/suitesparse subdirectory for SuiteSparse headers (PR #3391)
      • PythonBundle should perform 1 single "pip check" instead of each python package repeating it (issue #3418)
    • enhancements
      • Set CMake hints for path to Python installation if it's direct or indirect dependency (when using CMake >= 3.12) (PR #3282)
      • enhance LLVM easyblock for compilation of clang/flang + other llvm-projects (PR #3373)
      • enhance custom easyblock for GCC to use with-arch option for nvptx with 13.1+ (PR #3396)
    • updates
      • ...
    • changes
      • ...
    • new
      • ...
    • EasyBuild 5.0
      • Use prefer_ebpythonprefixes flag to use EBPYTHONPREFIXES (PR #3343)
      • enhance CMakeMake easyblock to check whether correct Python installation was picked up by CMake (PR #3399)
      • use PYPI_SOURCE constant in generic PythonPackage easyblock + custom easyblock for Python (PR #3410)
      • use run_shell_cmd in custom easyblock for ABAQUS (PR #3414)
      • remove (20) unused software-specific easyblocks (PR #3423)
      • remove unused easyblock for BamTools (PR #3424)

  • easyconfigs (open PRs + issues)

    • bug fixes/reports
      • ...
    • enhancements
      • ...
    • (noteworthy) new software
      • ...
    • software updates
    • changes
      • reorganize Flax/JAX stack in 2023a: scvi-tools, Flax, Optax, jax, absl-py (PRs #21038, #21039)

2024b common toolchains

  • GCC 14.2 as a base (see easyconfigs PR #21114)
  • we should define candidate toolchains using latest version of all components
    • are newer versions of Intel components compared to intel/2024a already available?
  • should we consider replacing intel with Clang-based toolchain as 2nd common toolchain next to foss?
    • Fortran frontend in Clang is becoming mature enough?

Q&A / others

Clone this wiki locally