Skip to content

Latest commit

 

History

History
39 lines (26 loc) · 3.63 KB

Build-new-external-software-stack.md

File metadata and controls

39 lines (26 loc) · 3.63 KB
Raw

Building and deploying external packages

Preliminary

You'll want to do the following logged onto daq.fnal.gov as user dunedaq. Ideally you'll be logged on in a manner such that it's unlikely your ssh connection will broken; while strides have been made in getting the build-ext.sh script to be able to pick up where it left off, it's generally better to be able to run build-ext.sh in one go. Note that this takes about 2-3 hours.

Build setup and start

In a nutshell, all you need to do is run the build-ext.sh script inside a container based on the ghcr.io/dune-daq/alma9-spack:latest image. To provide a bit more detail, you'll want to do the following once you're logged into daq.fnal.gov as dunedaq:

  1. Check whether there are already externals installed in /home/nfs/dunedaq/docker-scratch/cvmfs_dunedaq/spack/externals/ext-v${EXT_VERSION}/spack-${SPACK_VERSION} (*), and if so, that you know why they're already there.
  2. Create a directory which will be the base of operations for your work, if you don't already have one
  3. Inside that directory, git clone https://github.com/DUNE-DAQ/daq-release
  4. Launch a container using the example at the top of the build-ext.sh script as a guide. Note that <location of local area for installation> here would be /home/nfs/dunedaq/docker-scratch/cvmfs_dunedaq (as of Jul-25-2024)
  5. Run /daq-release/scripts/spack/build-ext.sh if you want to build everything from scratch (which you do if this is your first time)
  6. or /daq-release/scripts/spack/build-ext.sh false if you want to resume an externals build (e.g. because your ssh connection got broken)

(*) Here, use ${EXT_VERSION} and ${SPACK_VERSION} as stand-ins for the actual externals version (e.g., 2.1) and Spack version (e.g., 0.22.0)

Once the build is complete

Once complete, the externals you've built will be located in /home/nfs/dunedaq/docker-scratch/cvmfs_dunedaq/ext-v${EXT_VERSION}/spack-${SPACK_VERSION}. Note that you'll need to get them copied from the local area on daq.fnal.gov to two separate locations: (1) an externals image in which the nightly build can be performed, and (2) onto cvmfs. Please confirm you can build a nightly in an externals image before altering cvmfs . In order to do so:

  1. Run the Build docker with slim externals GitHub Action. Make sure you add an argument to the "optional suffix for test-only externals image" field, otherwise you'll clobber the standard externals image used for the usual nightly. This Action usually takes very roughly an hour, sometimes a bit less.

  2. Create a temporary branch forked off of the develop branch where you can modify the relevant nightly workflow YAML file (i.e., the nightly you plan to base off the externals you've built). The, modify the file so that instead of using the standard externals image, it uses the test image you created. E.g., if you provided T as an argument to "Build docker with slim externals", you'd stick a T at the end of the externals image referred to in the workflow file.

  3. Run the workflow off the temporary branch, and make sure that you provide a nightly tag prefix so you don't clobber the standard nightly. Also select yes for whether to deploy the release to cvmfs if it builds correctly.

  4. If it does, in fact, build successfully, now you can update the externals area on the actual cvmfs. Details on how to do that are here.