Skip to content

Run a quick rebuild to test EESSI-extend and eb_hooks.py in CI #99

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Run a quick rebuild to test EESSI-extend and eb_hooks.py in CI #99

Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
2ea186d
Run a quick rebuild to test EESSI-extend and eb_hooks.py
ocaisa Oct 3, 2025
0ae6d20
Merge branch 'main' into test_build_eessi_extend
ocaisa Oct 3, 2025
5b274fb
Make sure we check the EB CUDA settings for all cases
ocaisa Oct 6, 2025
6afde39
Merge branch 'test_build_eessi_extend' of github.com:ocaisa/software-…
ocaisa Oct 6, 2025
9d94539
Add some debug echo
ocaisa Oct 6, 2025
b1b5203
Tweak
ocaisa Oct 6, 2025
0adaebd
Tweak
ocaisa Oct 6, 2025
4218a7a
Tweak
ocaisa Oct 6, 2025
9013873
Add some more verbose checks
ocaisa Oct 6, 2025
a2e83db
No accelerator installations available for 2025.06 yet
ocaisa Oct 6, 2025
b979755
Make sure it always run for each matrix entry
ocaisa Oct 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 53 additions & 23 deletions .github/workflows/tests_eessi_extend_module.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,19 @@ jobs:
strategy:
fail-fast: false
matrix:
EESSI_VERSION:
- '2023.06'
#- '2025.06'
include:
- eessi_version: '2023.06'
rebuild_software: "BCFtools-1.18-GCC-12.3.0.eb"
# - eessi_version: '2025.06'
# rebuild_software: "make-4.4.1-GCCcore-13.3.0.eb"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this one commented out?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is only because there is not, as yet, any existing accelerator path for 2025.06, until the accel subdirectory exists this specific CI file will fail (the override does not work when the path doesn't exist, see https://github.com/EESSI/software-layer-scripts/actions/runs/18274775270/job/52024259482#step:6:119)

steps:
- name: Check out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Mount EESSI CernVM-FS repository
uses: eessi/github-action-eessi@v3
with:
eessi_stack_version: ${{matrix.EESSI_VERSION}}
eessi_stack_version: ${{matrix.eessi_version}}
use_eessi_module: true

- name: Install the EESSI-extend shipped with the repository
Expand All @@ -38,7 +40,7 @@ jobs:
check_disallowed_env_prefix EASYBUILD_

# Load the EESSI module
module load EESSI/${{matrix.EESSI_VERSION}}
module load EESSI/${{matrix.eessi_version}}
check_disallowed_env_prefix EASYBUILD_

# Load EasyBuild to install EESSI-extend
Expand All @@ -50,8 +52,12 @@ jobs:

# Verify that we can pick the installed version up
module use $MY_INSTALLATION_PATH/modules/all
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild
module load EESSI-extend/${{matrix.eessi_version}}-easybuild
echo $EBROOTEESSIMINEXTEND | grep $MY_INSTALLATION_PATH || { echo "ERROR: Installed version of EESSI-extend not picked up (loaded $EBROOTEESSIMINEXTEND)" >&2; exit 1; }
# Do a test rebuild (with default and local hooks)
eb --rebuild ${{matrix.rebuild_software}}
eb --rebuild --hooks=./eb_hooks.py ${{matrix.rebuild_software}}
# Proceed with unload and checking that
module unload EESSI-extend
# That should have unset all EasyBuild envvars (including EASYBUILD_PREFIX)
check_disallowed_env_prefix EASYBUILD_
Expand All @@ -68,19 +74,20 @@ jobs:

# Let's start from a clean slate
module purge
module load EESSI/${{matrix.EESSI_VERSION}}
module load EESSI/${{matrix.eessi_version}}
# Access the installed EESSI-extend
module use "$MY_INSTALLATION_PATH"/modules/all
check_disallowed_env_prefix EASYBUILD_

# Configure for CVMFS install
echo "Checking EESSI-extend CVMFS install"
export EESSI_CVMFS_INSTALL=1
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild
module load EESSI-extend/${{matrix.eessi_version}}-easybuild
# check some common EasyBuild settings
check_env_var "EASYBUILD_READ_ONLY_INSTALLDIR" "1"
check_env_var "EASYBUILD_ALLOW_LOADED_MODULES" "EasyBuild,EESSI-extend"
check_env_var "EASYBUILD_SYSROOT" "$EESSI_EPREFIX"
check_env_var "EASYBUILD_HOOKS" "/cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}/init/easybuild/eb_hooks.py"
check_env_var "EASYBUILD_HOOKS" "/cvmfs/software.eessi.io/versions/${{matrix.eessi_version}}/init/easybuild/eb_hooks.py"
check_env_var "EASYBUILD_RPATH" "1"
check_env_var "EASYBUILD_FILTER_ENV_VARS" "LD_LIBRARY_PATH"
# check some specific ones
Expand All @@ -92,7 +99,7 @@ jobs:
check_env_var "EASYBUILD_FAIL_ON_MOD_FILES_GCCCORE" "1"
check_env_var "EASYBUILD_LOCAL_VAR_NAMING_CHECK" "error"
# Check some EESSI version specific settings
if [[ "${{ matrix.EESSI_VERSION }}" != "2023.06" ]]; then
if [[ "${{ matrix.eessi_version }}" != "2023.06" ]]; then
check_env_var "EASYBUILD_PREFER_PYTHON_SEARCH_PATH" "EBPYTHONPREFIXES"
check_env_var "EASYBUILD_MODULE_SEARCH_PATH_HEADERS" "include_paths"
check_env_var "EASYBUILD_SEARCH_PATH_CPP_HEADERS" "include_paths"
Expand All @@ -105,8 +112,9 @@ jobs:
unset EESSI_CVMFS_INSTALL

# Now configure for a site
echo "Checking EESSI-extend site install"
export EESSI_SITE_INSTALL=1
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild
module load EESSI-extend/${{matrix.eessi_version}}-easybuild
# check some specific envvars
check_env_var "EASYBUILD_INSTALLPATH" "$EESSI_SITE_SOFTWARE_PATH"
check_env_var "EASYBUILD_UMASK" "022"
Expand All @@ -118,8 +126,9 @@ jobs:
unset EESSI_SITE_INSTALL

# Now for a project
echo "Checking EESSI-extend project install"
export EESSI_PROJECT_INSTALL="$MY_INSTALLATION_PATH"
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild
module load EESSI-extend/${{matrix.eessi_version}}-easybuild
# check some specific envvars
export EXPECTED_INSTALLATION_PATH="$MY_INSTALLATION_PATH/versions/$EESSI_VERSION/software/linux/$EESSI_SOFTWARE_SUBDIR"
check_env_var "EASYBUILD_INSTALLPATH" "$EXPECTED_INSTALLATION_PATH"
Expand All @@ -133,9 +142,10 @@ jobs:
unset EESSI_PROJECT_INSTALL

# Now for a user
echo "Checking EESSI-extend user install"
export EESSI_USER_INSTALL="$MY_INSTALLATION_PATH/$USER"
mkdir -p $EESSI_USER_INSTALL # must exist
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild
module load EESSI-extend/${{matrix.eessi_version}}-easybuild
# check some specific envvars
export EXPECTED_INSTALLATION_PATH="$MY_INSTALLATION_PATH/$USER/versions/$EESSI_VERSION/software/linux/$EESSI_SOFTWARE_SUBDIR"
check_env_var "EASYBUILD_INSTALLPATH" "$EXPECTED_INSTALLATION_PATH"
Expand Down Expand Up @@ -163,66 +173,86 @@ jobs:
# Let's start from a clean slate
module purge
export EESSI_ACCELERATOR_TARGET_OVERRIDE=$STORED_EESSI_ACCELERATOR_TARGET_OVERRIDE
module load EESSI/${{matrix.EESSI_VERSION}}
module load EESSI/${{matrix.eessi_version}}
# Access the installed EESSI-extend
module use "$MY_INSTALLATION_PATH"/modules/all
check_disallowed_env_prefix EASYBUILD_

# Configure for CVMFS install
echo "Checking EESSI-extend CVMFS install"
export EESSI_CVMFS_INSTALL=1
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild
module load EESSI-extend/${{matrix.eessi_version}}-easybuild
check_env_var "EASYBUILD_INSTALLPATH" "$EESSI_SOFTWARE_PATH" # installation path should be the same unless we ask for an explicit GPU installation
check_env_var "EESSI_ACCELERATOR_TARGET" "$STORED_EESSI_ACCELERATOR_TARGET_OVERRIDE"
check_env_var "EASYBUILD_CUDA_COMPUTE_CAPABILITIES" "$STORED_CUDA_CC"
module unload EESSI-extend
export EESSI_ACCELERATOR_INSTALL=1
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild # reload for an actual GPU installation
module load EESSI-extend/${{matrix.eessi_version}}-easybuild # reload for an actual GPU installation
check_env_var "EASYBUILD_INSTALLPATH" "${EESSI_SOFTWARE_PATH}/${EESSI_ACCELERATOR_TARGET_OVERRIDE}"
check_env_var "EESSI_ACCELERATOR_TARGET" "$STORED_EESSI_ACCELERATOR_TARGET_OVERRIDE"
check_env_var "EASYBUILD_CUDA_COMPUTE_CAPABILITIES" "$STORED_CUDA_CC"
# unload and make sure the environment is clean again
module unload EESSI-extend
check_disallowed_env_prefix EASYBUILD_
unset EESSI_ACCELERATOR_INSTALL
unset EESSI_CVMFS_INSTALL

# Now configure for a site
echo "Checking EESSI-extend site install"
export EESSI_SITE_INSTALL=1
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild
module load EESSI-extend/${{matrix.eessi_version}}-easybuild
check_env_var "EASYBUILD_INSTALLPATH" "$EESSI_SITE_SOFTWARE_PATH" # installation path should be the same unless we ask for an explicit GPU installation
check_env_var "EESSI_ACCELERATOR_TARGET" "$STORED_EESSI_ACCELERATOR_TARGET_OVERRIDE"
check_env_var "EASYBUILD_CUDA_COMPUTE_CAPABILITIES" "$STORED_CUDA_CC"
module unload EESSI-extend
export EESSI_ACCELERATOR_INSTALL=1
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild # reload for an actual GPU installation
module load EESSI-extend/${{matrix.eessi_version}}-easybuild # reload for an actual GPU installation
check_env_var "EASYBUILD_INSTALLPATH" "${EESSI_SITE_SOFTWARE_PATH}/${EESSI_ACCELERATOR_TARGET_OVERRIDE}"
check_env_var "EESSI_ACCELERATOR_TARGET" "$STORED_EESSI_ACCELERATOR_TARGET_OVERRIDE"
check_env_var "EASYBUILD_CUDA_COMPUTE_CAPABILITIES" "$STORED_CUDA_CC"
# unload and make sure the environment is clean again
module unload EESSI-extend
check_disallowed_env_prefix EASYBUILD_
unset EESSI_ACCELERATOR_INSTALL
unset EESSI_SITE_INSTALL

# Now for a project
echo "Checking EESSI-extend project install"
export EESSI_PROJECT_INSTALL="$MY_INSTALLATION_PATH"
export EXPECTED_INSTALLATION_PATH="$MY_INSTALLATION_PATH/versions/$EESSI_VERSION/software/linux/$EESSI_SOFTWARE_SUBDIR"
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild
module load EESSI-extend/${{matrix.eessi_version}}-easybuild
check_env_var "EASYBUILD_INSTALLPATH" "$EXPECTED_INSTALLATION_PATH" # installation path should be the same unless we ask for an explicit GPU installation
check_env_var "EESSI_ACCELERATOR_TARGET" "$STORED_EESSI_ACCELERATOR_TARGET_OVERRIDE"
check_env_var "EASYBUILD_CUDA_COMPUTE_CAPABILITIES" "$STORED_CUDA_CC"
module unload EESSI-extend
export EESSI_ACCELERATOR_INSTALL=1
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild # reload for an GPU actual installation
module load EESSI-extend/${{matrix.eessi_version}}-easybuild # reload for an GPU actual installation
check_env_var "EASYBUILD_INSTALLPATH" "$EXPECTED_INSTALLATION_PATH" # installation path should be the same for project case
check_env_var "EESSI_ACCELERATOR_TARGET" "$STORED_EESSI_ACCELERATOR_TARGET_OVERRIDE"
check_env_var "EASYBUILD_CUDA_COMPUTE_CAPABILITIES" "$STORED_CUDA_CC"
# unload and make sure the environment is clean again
module unload EESSI-extend
check_disallowed_env_prefix EASYBUILD_
unset EESSI_ACCELERATOR_INSTALL
unset EESSI_PROJECT_INSTALL

# Now for a user
echo "Checking EESSI-extend user install"
export EESSI_USER_INSTALL="$MY_INSTALLATION_PATH/$USER"
mkdir -p $EESSI_USER_INSTALL # must exist
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild
module load EESSI-extend/${{matrix.eessi_version}}-easybuild
# check some specific envvars
export EXPECTED_INSTALLATION_PATH="$MY_INSTALLATION_PATH/$USER/versions/$EESSI_VERSION/software/linux/$EESSI_SOFTWARE_SUBDIR"
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild
module load EESSI-extend/${{matrix.eessi_version}}-easybuild
check_env_var "EASYBUILD_INSTALLPATH" "$EXPECTED_INSTALLATION_PATH" # installation path should be the same unless we ask for an explicit GPU installation
check_env_var "EESSI_ACCELERATOR_TARGET" "$STORED_EESSI_ACCELERATOR_TARGET_OVERRIDE"
check_env_var "EASYBUILD_CUDA_COMPUTE_CAPABILITIES" "$STORED_CUDA_CC"
module unload EESSI-extend
export EESSI_ACCELERATOR_INSTALL=1
module load EESSI-extend/${{matrix.EESSI_VERSION}}-easybuild # reload for an actual GPU installation
module load EESSI-extend/${{matrix.eessi_version}}-easybuild # reload for an actual GPU installation
check_env_var "EASYBUILD_INSTALLPATH" "$EXPECTED_INSTALLATION_PATH" # installation path should be the same for user case
check_env_var "EESSI_ACCELERATOR_TARGET" "$STORED_EESSI_ACCELERATOR_TARGET_OVERRIDE"
check_env_var "EASYBUILD_CUDA_COMPUTE_CAPABILITIES" "$STORED_CUDA_CC"
# unload and make sure the environment is clean again
module unload EESSI-extend
check_disallowed_env_prefix EASYBUILD_
Expand Down