Skip to content

EasyBuild Sprint 2020a toolchains

Jump to bottom
Kenneth Hoste edited this page Mar 12, 2020 · 25 revisions

What?

Starting to use a brand new toolchain is always a challenge, since the limited number of EasyConfigs mean you need to develop a lot yourself. This sprint synchronizes that effort amongst the community, so that together we can kickstart the development of GCCcore-9.x, foss-2020a, intel-2020a based EasyConfigs. Hopefully, this will result in a large number of EasyConfigs being available quickly after the release of the 2020a toolchains.

When?

The sprint will take place on Tuesday 17th and Thursday 19th of March 2020, 9:00-13:00h CET. During this time, a large number of maintainers will be available for reviewing and accepting PRs.

The idea of having two separate mornings is that straightforward EasyConfigs can go through the full review cycle on the 17th. For more difficult ones, you can get some feedback on the 17th, work on it in the meantime, and try to finalize the PR on the 19th.

How to participate?

  • Before the sprint: make sure to prepare your environment so you can participate (see the 'How to prepare?' section)
  • Check out the wishlist and choose an a software package you want to work on (i.e. make an EasyConfig for). If you want to work on something that is not on the wishlist but that is relevant for your site, just ask one of the maintainers to add it to the wishlist.
  • Ask one of the maintainers to add your name to the software package that you will work on. This avoids having two people working on the same EasyConfig at the same time.
  • Develop your EasyConfig
  • Use --new-pr to submit a pull request. In addition, make sure it targets the 2020a branch by adding --pr-target-branch=2020a

What to prepare before the sprint?

It is advisable to set up an environment beforehand that

In order to do create this setup, please take the following steps:

If you do not have a fork yet

# pick an installation prefix (adjust as you like)
INSTALL_PREFIX=$(mktemp -d $HOME/EasyBuild-XXXXXX)
# download script
curl -O https://raw.githubusercontent.com/easybuilders/easybuild-framework/master/easybuild/scripts/install-EasyBuild-develop.sh
# Remove vsc-based lines, that repo is no longer needed for EasyBuild 4.x
sed -i '/^.*vsc-.*$/d' install-EasyBuild-develop.sh
# Change it so it targets the 2020a branch for the EasyConfigs repository:
sed -i 's/github_clone_branch "easybuild-easyconfigs" "develop"/github_clone_branch "easybuild-easyconfigs" "2020a"/g' install-EasyBuild-develop.sh
# run downloaded script, specifying *your* GitHub username and the installation prefix
bash install-EasyBuild-develop.sh GITHUB_USERNAME $INSTALL_PREFIX
# update $MODULEPATH via 'module use', and load the module
module use $INSTALL_PREFIX/modules
module load EasyBuild-develop
eb --version  ## This should ensure you have a reasonable instance of EasyBuild
  • Chance directory to the easybuild-easyconfigs folder
cd $INSTALL_PREFIX/easybuild-easyconfigs
  • Check that it targets the right branch, i.e. git branch should return * 2020a (if not: run git checkout 2020a)

If you already have a fork

  • Use the (unmodified) install-EasyBuild-develop.sh script to install the repositories
# pick an installation prefix (adjust as you like)
INSTALL_PREFIX=$(mktemp -d $HOME/EasyBuild-XXXXXX)
# download script
curl -O https://raw.githubusercontent.com/easybuilders/easybuild-framework/master/easybuild/scripts/install-EasyBuild-develop.sh
# Remove vsc-based lines, that repo is no longer needed for EasyBuild 4.x
sed -i '/^.*vsc-.*$/d' install-EasyBuild-develop.sh
# run downloaded script, specifying *your* GitHub username and the installation prefix
bash install-EasyBuild-develop.sh GITHUB_USERNAME $INSTALL_PREFIX
# update $MODULEPATH via 'module use', and load the module
module use $INSTALL_PREFIX/modules
module load EasyBuild-develop
eb --version  ## This should ensure you have a reasonable instance of EasyBuild
  • Chance directory to the easybuild-easyconfigs folder (cd $INSTALL_PREFIX/easybuild-easyconfigs)
  • Checkout the 2020a branch (git checkout 2020a)

Configuring the EasyBuild installation

The above steps have created an installation of a development version of EasyBuild that you will use throughout the sprint. You may however want to mimic (part of) the configuration of your production installation of EasyBuild. One option is to manually modify the installed $INSTALL_PREFIX/modules/EasyBuild-develop modulefile to export desired EASYBUILD_* environment variables.

You can check your configuration using

eb --show-config

Configuring the GitHub integration

To set up the GitHub integration, please follow the official EasyBuild documentation on that. You can check it has been set up correctly by trying

eb --check-github

(if you haven't configured a default github user for your EasyBuild installation, don't forget to add --github-user=<username>)

Install the 2020a-based toolchains

Depending on what you want to participate in, you may want to install the foss-2020a or intel-2020a toolchains. Currently (12 March), the newest toolchain definitions are foss-2020a-rc3 and intel-2020a-rc1. There may still be slight changes in the toolchain before release (e.g. GCCcore-9.2.0 may be bumped to GCCcore-9.3.0), but these are generally not expected to change the functionality of the EasyConfigs.

Since there is no CUDA version that currently supports GCCcore-9.x, you do not need to install a newer fosscuda - development for fosscuda will likely not happen within this sprint.

Clone this wiki locally