Description

This module can be used to setup and install Spack on a VM. To actually run Spack commands to install other software use the spack-execute module.

This module generates a script that performs the following:

Install system dependencies needed for Spack
Clone Spack into a predefined directory
Check out a specific version of Spack

There are several options on how to consume the outputs of this module:

Important

Breaking changes between after v1.21.0. spack-install module replaced by spack-setup and spack-execute modules. Details Below

Examples

`use` `spack-setup` with `spack-execute`

This will prepend the spack-setup script to the spack-execute commands.

  - id: spack-setup
    source: community/modules/scripts/spack-setup

  - id: spack-build
    source: community/modules/scripts/spack-execute
    use: [spack-setup]
    settings:
      commands: |
        spack install gcc@10.3.0 target=x86_64

  - id: builder
    source: modules/compute/vm-instance
    use: [network1, spack-build]

`use` `spack-setup` with `vm-instance` or Slurm module

This will run spack-setup scripts on the downstream compute resource.

  - id: spack-setup
    source: community/modules/scripts/spack-setup

  - id: spack-installer
    source: modules/compute/vm-instance
    use: [network1, spack-setup]

OR

  - id: spack-setup
    source: community/modules/scripts/spack-setup

  - id: slurm_controller
    source: community/modules/scheduler/schedmd-slurm-gcp-v5-controller
    use: [network1, partition1, spack-setup]

Build `starup-script` with `spack-runner` output

This will use the generated spack-setup script as one step in startup-script.

  - id: spack-setup
    source: community/modules/scripts/spack-setup

  - id: startup-script
    source: modules/scripts/startup-script
    settings:
      runners:
      - $(spack-setup.spack-runner)
      - type: shell
        destination: "my-script.sh"
        content: echo 'hello world'

  - id: workstation
    source: modules/compute/vm-instance
    use: [network1, startup-script]

To see a full example of this module in use, see the hpc-slurm-gromacs.yaml example.

Environment Setup

Activating Spack

Spack installation produces a setup script that adds spack to your PATH as well as some other command-line integration tools. This script can be found at <install path>/share/spack/setup-env.sh. This script will be automatically added to bash startup by any machine that runs the spack_runner.

If you have multiple machines that all want to use the same shared Spack installation you can just have both machines run the spack_runner.

Managing Spack Python dependencies

Spack is configured with SPACK_PYTHON to ensure that Spack itself uses a Python virtual environment with a supported copy of Python with the package google-cloud-storage pre-installed. This enables Spack to use mirrors and build caches on Google Cloud Storage. It does not configure Python packages inside Spack virtual environments. If you need to add more Python dependencies for Spack itself, use the spack python command:

sudo -i spack python -m pip install package-name

Spack Permissions

System `spack` user is created - Default

By default this module will create a spack linux user and group with consistent UID and GID. This user and group will own the Spack installation. To allow a user to manually add Spack packages to the system Spack installation, you can add the user to the spack group:

sudo usermod -a -G spack <username>

Log out and back in so the group change will take effect, then <username> will be able to call spack install <package>.

Note

A background persistent SSH connections may prevent the group change from taking effect.

You can use the system_user_name, system_user_uid, and system_user_gid to customize the name and ids of the system user. While unlikely, it is possible that the default system_user_uid or system_user_gid could conflict with existing UIDs.

Use and existing user

Alternatively, if system_user_name is a user already on the system, then this existing user will be used for Spack installation.

OS Login User

If OS Login is enabled (default for most Cluster Toolkit modules) then you can provide an OS Login user name:

  - id: spack-setup
    source: community/modules/scripts/spack-setup
    settings:
      system_user_name: username_company_com

This will work even if the user has not yet logged onto the machine. When the specified user does log on to the machine they will be able to call spack install without any further configuration.

Pre-configured user

You can also use a startup script to configure a user:

  - id: spack-setup
    source: community/modules/scripts/spack-setup
    settings:
      system_user_name: special-user

  - id: startup
    source: modules/scripts/startup-script
    settings:
      runners:
        - type: shell
          destination: "create_user.sh"
          content: |
            #!/bin/bash
            sudo useradd -u 799 special-user
            sudo groupadd -g 922 org-group
            sudo usermod -g org-group special-user
        - $(spack-setup.spack_runner)

  - id: spack-vms
    source: modules/compute/vm-instance
    use: [network1, startup]
    settings:
      name_prefix: spack-vm
      machine_type: n2d-standard-2
      instance_count: 5

Chaining spack installations

If there is a need to have a non-root user to install spack packages it is recommended to create a separate installation for that user and chain Spack installations (Spack docs).

Steps to chain Spack installations:

Get the version of the system Spack:

$ spack --version

0.20.0 (e493ab31c6f81a9e415a4b0e0e2263374c61e758)
#       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Note commit hash and use in next step

Clone a new spack installation:

git clone -c feature.manyFiles=true https://github.com/spack/spack.git <new_location>/spack
git -C <new_location>/spack checkout <commit_hash>

Point the new Spack installation to the system Spack installation. Create a file at <new_location>/spack/etc/spack/upstreams.yaml with the following contents:
```
upstreams:
  spack-instance-1:
    install_tree: /sw/spack/opt/spack/
```
Add the following line to your .bashrc to make sure the new spack is in your PATH.
```
. <new_location>/spack/share/spack/setup-env.sh
```

Deprecations and Breaking Changes

The old spack-install module has been replaced by the spack-setup and spack-execute modules. Generally this change strives to allow for a more flexible definition of a Spack build by using native Spack commands.

For every deprecated variable from spack-install there is documentation on how to perform the equivalent action using commands and data_files. The documentation can be found on the inputs table below.

Below is a simple example of the same functionality shown before and after the breaking changes.

  # Before
  - id: spack-install
    source: community/modules/scripts/spack-install
    settings:
      install_dir: /sw/spack
      compilers:
      - gcc@10.3.0 target=x86_64
      packages:
      - intel-mpi@2018.4.274%gcc@10.3.0

- id: spack-startup
    source: modules/scripts/startup-script
    settings:
      runners:
      - $(spack.install_spack_deps_runner)
      - $(spack.install_spack_runner)

  # After
  - id: spack-setup
    source: community/modules/scripts/spack-setup
    settings:
      install_dir: /sw/spack

  - id: spack-execute
    source: community/modules/scripts/spack-execute
    use: [spack-setup]
    settings:
      commands: |
        spack install gcc@10.3.0 target=x86_64
        spack load gcc@10.3.0 target=x86_64
        spack compiler find --scope site
        spack install intel-mpi@2018.4.274%gcc@10.3.0

- id: spack-startup
    source: modules/scripts/startup-script
    settings:
      runners:
      - $(spack-execute.spack-runner)

Although the old spack-install module will no longer be maintained, it is still possible to use the old module in a blueprint by referencing an old version from GitHub. Note the source line in the following example.

  - id: spack-install
    source: github.com/GoogleCloudPlatform/hpc-toolkit//community/modules/scripts/spack-install?ref=v1.22.1&depth=1

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Requirements

Name	Version
terraform	>= 1.0.0
google	>= 4.42
local	>= 2.0.0

Providers

Name	Version
google	>= 4.42
local	>= 2.0.0

Modules

Name	Source	Version
startup_script	github.com/GoogleCloudPlatform/hpc-toolkit//modules/scripts/startup-script	v1.39.0&depth=1

Resources

Name	Type
google_storage_bucket.bucket	resource
local_file.debug_file_shell_install	resource

Inputs

Name	Description	Type	Default	Required
chmod_mode	`chmod` to apply to the Spack installation. Adds group write by default. Set to `""` (empty string) to prevent modification. For usage information see: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/file_module.html#parameter-mode	`string`	`"g+w"`	no
configure_for_google	When true, the spack installation will be configured to pull from Google's Spack binary cache.	`bool`	`true`	no
deployment_name	Name of deployment, used to name bucket containing startup script.	`string`	n/a	yes
install_dir	Directory to install spack into.	`string`	`"/sw/spack"`	no
labels	Key-value pairs of labels to be added to created resources.	`map(string)`	n/a	yes
project_id	Project in which the HPC deployment will be created.	`string`	n/a	yes
region	Region to place bucket containing startup script.	`string`	n/a	yes
spack_profile_script_path	Path to the Spack profile.d script. Created by this module	`string`	`"/etc/profile.d/spack.sh"`	no
spack_ref	Git ref to checkout for spack.	`string`	`"v0.20.0"`	no
spack_url	URL to clone the spack repo from.	`string`	`"https://github.com/spack/spack"`	no
spack_virtualenv_path	Virtual environment path in which to install Spack Python interpreter and other dependencies	`string`	`"/usr/local/spack-python"`	no
system_user_gid	GID used when creating system user group. Ignored if `system_user_name` already exists on system. Default of 1104762903 is arbitrary.	`number`	`1104762903`	no
system_user_name	Name of system user that will perform installation of Spack. It will be created if it does not exist.	`string`	`"spack"`	no
system_user_uid	UID used when creating system user. Ignored if `system_user_name` already exists on system. Default of 1104762903 is arbitrary.	`number`	`1104762903`	no

Outputs

Name	Description
controller_startup_script	Spack installation script, duplicate for SLURM controller.
gcs_bucket_path	Bucket containing the startup scripts for spack, to be reused by spack-execute module.
spack_path	Path to the root of the spack installation
spack_profile_script_path	Path to the Spack profile.d script.
spack_runner	Runner to be used with startup-script module or passed to spack-execute module. - installs Spack dependencies - installs Spack - generates profile.d script to enable access to Spack This is safe to run in parallel by multiple machines. Use in place of deprecated `setup_spack_runner`.
startup_script	Spack installation script.
system_user_name	The system user used to install Spack. It can be reused by spack-execute module to install spack packages.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

Description

Examples

`use` `spack-setup` with `spack-execute`

`use` `spack-setup` with `vm-instance` or Slurm module

Build `starup-script` with `spack-runner` output

Environment Setup

Activating Spack

Managing Spack Python dependencies

Spack Permissions

System `spack` user is created - Default

Use and existing user

OS Login User

Pre-configured user

Chaining spack installations

Deprecations and Breaking Changes

License

Requirements

Providers

Modules

Resources

Inputs

Outputs

Files

README.md

Latest commit

History

README.md

File metadata and controls

Description

Examples

use spack-setup with spack-execute

use spack-setup with vm-instance or Slurm module

Build starup-script with spack-runner output

Environment Setup

Activating Spack

Managing Spack Python dependencies

Spack Permissions

System spack user is created - Default

Use and existing user

OS Login User

Pre-configured user

Chaining spack installations

Deprecations and Breaking Changes

License

Requirements

Providers

Modules

Resources

Inputs

Outputs

`use` `spack-setup` with `spack-execute`

`use` `spack-setup` with `vm-instance` or Slurm module

Build `starup-script` with `spack-runner` output

System `spack` user is created - Default