Skip to content

Commit

Permalink
Make HPE EX/Apollo more prominent in cray.html (chapel-lang#24656)
Browse files Browse the repository at this point in the history 
[reviewed by @jabraham17 and @jhh67]

While reviewing the website for the 2.0 release today, we realized that
HPE Cray EX isn't highlighted as a platform on our download page
alongside XC, which is increasingly being put to bed. Updating the
website is easy, but most of the material is here in the platform docs,
which was still very XC-centric in terms of ordering of material.

Here, I've taken a pass to try and bring HPE EX and Apollo more to the
forefront relative to Cray XC and CS, while also touching up some other
things that caught my eye. I think there's more we could/should do here
to bring HPE systems to the forefront and give them equal time and
attention to the historical Cray systems (like, for example, renaming
this file), but am currently imagining that to be post-2.0 work.
  • Loading branch information
@bradcray
bradcray authored Mar 20, 2024
2 parents 8a69c4b + 51a968c commit 62c8ed5
Showing 1 changed file with 86 additions and 78 deletions.
164 changes: 86 additions & 78 deletions doc/rst/platforms/cray.rst
View file
Original file line number Diff line number Diff line change
@@ -1,34 +1,33 @@
.. _readme-cray:

============================
Using Chapel on Cray Systems
============================
================================
Using Chapel on HPE Cray Systems
================================

The following information is assembled to help Chapel users get up and running
on HPE Cray\ |reg| systems including the Cray XC\ |trade| and
HPE Cray EX series systems.
The following information is assembled to help Chapel users get up and
running on HPE Cray systems including the HPE Cray EX and Cray XC
series systems.

.. contents::


-------------------------------------------------------------
Getting Started with Chapel on Cray XC or HPE Cray EX Systems
-------------------------------------------------------------
--------------------------------------------------------
Getting Started with Chapel on HPE Cray EX or XC Systems
--------------------------------------------------------

Chapel is available as a module for Cray XC and HPE Cray EX systems.
When it is installed on your system, you do not need to build Chapel
from the source release (though you can). With either the traditional
Tcl-based module system or the newer Lua-based Lmod module system, to
use Chapel with the default settings and confirm it is correctly
installed, do the following:
Chapel is available as a module for HPE Cray EX and XC systems. When
it is installed on your system, you do not need to build Chapel from
the source release (though you can). Using the module systems on such
platforms, you can use Chapel with the default settings and confirm it
is correctly installed, as follows:

1) Ensure this required module is loaded. Normally it will be loaded
for you, but under some circumstances you may need to load or
restore it yourself:

.. code-block:: sh
PrgEnv-cray or PrgEnv-gnu
PrgEnv-gnu or PrgEnv-cray
2) Load the Chapel module::
Expand All @@ -46,63 +45,74 @@ installed, do the following:
./hello6-taskpar-dist -nl 4


This should be all that is necessary to use Chapel on a Cray XC or HPE
Cray EX system. If the installation setup by your system administrator
This should be all that is necessary to use Chapel on an HPE Cray EX or XC
system. If the installation setup by your system administrator
deviates from the default settings, or you are interested in other
configuration options, see `Using Chapel on a Cray System`_ below. If
configuration options, see `Using Chapel on an HPE Cray System`_ below. If
instead you wish to build Chapel from source, continue on to `Building
Chapel for a Cray System from Source`_ just below.
Chapel for an HPE Cray System from Source`_ just below.

Note that the Chapel module for HPE Cray EX systems does not yet have
the gasnet communication layer built into it. For multilocale execution
Note that the Chapel module for HPE Cray EX systems does not currently have
the GASNet communication layer built into it. For multilocale execution
on EX systems please use the ofi communication layer instead.

For information on obtaining and installing the Chapel module please
contact your system administrator.


----------------------------------------------
Getting Started with Chapel on Cray CS Systems
----------------------------------------------
-------------------------------------------------------------
Getting Started with Chapel on HPE Apollo and Cray CS Systems
-------------------------------------------------------------

On Cray CS systems, Chapel is not currently available as a module due
to the wide diversity of software packages that Cray CS customers may
choose to install on their system. For this reason, Chapel must be
built from source on Cray CS systems using the
`Building Chapel for a Cray System from Source`_ instructions just below.
On HPE Apollo and Cray CS systems, Chapel is not currently available
as a module due to the wide diversity of configurations that these
systems support. For this reason, Chapel must be built from source on
these systems using the `Building Chapel for an HPE Cray System from
Source`_ instructions just below.


---------------------------------------------
Building Chapel for a Cray System from Source
---------------------------------------------
--------------------------------------------------
Building Chapel for an HPE Cray System from Source
--------------------------------------------------

1) If using an XC or EX system, continue to step 2. If using a CS series
system, set ``CHPL_HOST_PLATFORM`` to ``cray-cs``.
1) Set ``CHPL_HOST_PLATFORM`` to the string representing your system
type.

For example:

.. code-block:: sh
export CHPL_HOST_PLATFORM=cray-cs
export CHPL_HOST_PLATFORM=hpe-cray-ex
These are the supported systems and strings. Note that these values
are used by default when building on the given systems. They can
also be set manually.
The following table lists the supported systems and strings. Note
that on HPE Cray EX and XC systems, these values should typically
be inferred automatically and not need to be set manually. That said,
there is also no downside to setting them manually. As with most
`CHPL_*` environment variables, the current set and inferred values
can be determined by running ``$CHPL_HOME/util/printchplenv``.

========= ==================
System CHPL_HOST_PLATFORM
========= ==================
CS series cray-cs
XC series cray-xc
EX series hpe-cray-ex
========= ==================
============= ==================
System CHPL_HOST_PLATFORM
============= ==================
EX series hpe-cray-ex
Apollo series hpe-apollo
XC series cray-xc
CS series cray-cs
============= ==================


2) Optionally, set the ``CHPL_LAUNCHER`` environment variable to indicate
how Chapel should launch jobs on your system:

======================================== =========================
On a Cray CS system, to... set CHPL_LAUNCHER to...
On an HPE Cray EX system, ... set CHPL_LAUNCHER to...
======================================== =========================
...queue jobs using SLURM (sbatch) slurm-srun
...run jobs interactively using PALS pals
======================================== =========================

======================================== =========================
On a HPE Apollo or Cray CS system, to... set CHPL_LAUNCHER to...
======================================== =========================
...run jobs interactively on your system gasnetrun_ibv
...queue jobs using SLURM (sbatch) slurm-gasnetrun_ibv
Expand All @@ -118,24 +128,20 @@ Building Chapel for a Cray System from Source
...queue jobs using SLURM (sbatch) slurm-srun
======================================== =========================

======================================== =========================
On an HPE Cray EX system, ... set CHPL_LAUNCHER to...
======================================== =========================
...queue jobs using SLURM (sbatch) slurm-srun
...run jobs interactively using PALS pals
======================================== =========================

You can also set CHPL_LAUNCHER to ``none`` if you prefer to manually
manage all queuing and job launch commands yourself.
On HPE Cray EX systems, ``CHPL_LAUNCHER`` defaults to ``slurm-srun``
if ``srun`` is in your path and ``none`` otherwise.

On Cray CS systems, ``CHPL_LAUNCHER`` defaults to ``gasnetrun_ibv``.
On HPE Apollo and Cray CS systems, ``CHPL_LAUNCHER`` defaults to
``gasnetrun_ibv``.

On Cray XC systems, ``CHPL_LAUNCHER`` defaults to ``aprun`` if
``aprun`` is in your path, ``slurm-srun`` if ``srun`` is in your path
and ``none`` otherwise.

On HPE Cray EX systems, ``CHPL_LAUNCHER`` defaults to ``slurm-srun``
if ``srun`` is in your path and ``none`` otherwise.
You can also set CHPL_LAUNCHER to ``none`` if you prefer to manually
manage all queuing and job launch commands yourself, though the
advantage of using a launcher is to make sure that Chapel maps
processes and threads to the appropriate target hardware units.

For more information on Chapel's launcher capabilities and options,
refer to :ref:`readme-launcher`.
Expand All @@ -144,20 +150,22 @@ Building Chapel for a Cray System from Source
3) Select the target compiler that Chapel should use when compiling
code for the compute node:

On a Cray CS series system, set the ``CHPL_TARGET_COMPILER`` environment
variable to indicate which compiler to use (and make sure that the compiler
is in your path).
On an HPE Apollo or Cray CS series system, set the
``CHPL_TARGET_COMPILER`` environment variable to indicate which
compiler to use (and make sure that the compiler is in your path).

=========================== ==============================
To request... set CHPL_TARGET_COMPILER to...
=========================== ==============================
...the GNU compiler (gcc) gnu (default)
...the LLVM/clang backend llvm (default)
...the GNU compiler (gcc) gnu
...the Clang compiler clang
...the Intel compiler (icc) intel
=========================== ==============================

On a Cray XC or HPE Cray EX system, ensure that you have one of the
following Programming Environment modules loaded to specify your
target compiler::
On an HPE Cray EX or Cray XC system, when using the C back-end,
ensure that you have one of the following Programming Environment
modules loaded to specify your target compiler::

PrgEnv-allinea (ARM only)
PrgEnv-cray
Expand All @@ -179,9 +187,9 @@ Building Chapel for a Cray System from Source
between any of these configurations without rebuilding.


-----------------------------
Using Chapel on a Cray System
-----------------------------
----------------------------------
Using Chapel on an HPE Cray System
----------------------------------

1) If you are working from a Chapel module:

Expand Down Expand Up @@ -219,10 +227,10 @@ Using Chapel on a Cray System
is not large enough. Increase the number of KVS entries by
setting env variable PMI_MAX_KVS_ENTRIES to a higher value.
Having the Cray PMI modules loaded when the program is compiled will
prevent this problem. We expect that eventually these modules will
be loaded by default on EX systems, but so far this has not been the
case.
Having the Cray PMI modules loaded when the program is compiled
will prevent this problem. We expect that eventually these modules
will be loaded by default on EX systems, but so far this has not
consistently been the case.


3) Compile your Chapel program. For example:
Expand Down Expand Up @@ -282,10 +290,10 @@ Using Chapel on a Cray System
Requests the program to be executed using two locales.


6) If your Cray system has compute nodes with varying numbers of
6) If your HPE Cray system has compute nodes with varying numbers of
cores, you can request nodes with at least a certain number of
cores using the variable ``CHPL_LAUNCHER_CORES_PER_LOCALE``. For
example, on a Cray system in which some compute nodes have 24 or
example, on a system in which some compute nodes have 24 or
more cores per compute node, you could request nodes with at least
24 cores using:

Expand All @@ -311,10 +319,10 @@ Using Chapel on a Cray System
between Chapel launchers and workload managers.


7) If your Cray system has compute nodes with varying numbers of CPUs
7) If your HPE Cray system has compute nodes with varying numbers of CPUs
per compute unit, you can request nodes with a certain number of
CPUs per compute unit using the variable ``CHPL_LAUNCHER_CPUS_PER_CU``.
For example, on a Cray XC series system with some nodes having at
For example, on a Cray XC system with some nodes having at
least 2 CPUs per compute unit, to request running on those nodes
you would use:

Expand Down Expand Up @@ -655,7 +663,7 @@ Known Constraints and Bugs
* Redirecting stdin when executing a Chapel program under PBS/qsub
may not work due to limitations of qsub.

* For XC and EX systems, there is a known issue with the Cray MPI
* For EX and XC systems, there is a known issue with the Cray MPI
release that causes some programs to assert and then hang during
exit. A workaround is to set the environment variable,
``MPICH_GNI_DYNAMIC_CONN`` to ``disabled``. Setting this environment
Expand Down

0 comments on commit 62c8ed5

Please sign in to comment.