Skip to content

Commit

Permalink
Fixed a dead link, some typos. Marked pages as 'new', boost page in s…
Browse files Browse the repository at this point in the history
…earch
  • Loading branch information
CallumWalley committed Dec 4, 2024
1 parent 859cd6f commit 3062743
Show file tree
Hide file tree
Showing 5 changed files with 35 additions and 43 deletions.
10 changes: 0 additions & 10 deletions docs/General/Announcements/.pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,5 @@
nav:
- Preparing_to_move_data_to_NeSI_long_term_storage.md
- Preparing_your_code_for_use_on_NeSIs_new_HPC_platform.md
- NeSI_Support_is_changing_tools.md
- Accessing_NeSI_Support_during_the_holiday_break.md
- Improved_data_management_and_efficient_use_of_NeSI_HPC_storage.md
- Improvements_to_Fair_Share_job_prioritisation_on_Maui.md
- Mahuika-Core_Dumps_generation_now_disabled_as_default.md
- Mahuikas_new_Milan_CPU_nodes_open_to_all_NeSI_users.md
- Maui_upgrade_is_complete.md
- New_capabilities_for_Machine_Learning_and_GPU_pricing_updates.md
- Slurm_upgrade_to_version_21-8.md
- Status_page_subscription_notification_changes.md
- University_of_Auckland_ANSYS_users.md
- Upcoming_webinar-Tips_for_making_the_most_of_Mahuikas_new_Milan_nodes.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
created: 2024-09-05

Check warning on line 2 in docs/General/Announcements/Preparing_to_move_data_to_NeSI_long_term_storage.md

View workflow job for this annotation

GitHub Actions / Check page meta

meta_unexpected_key

Unexpected parameter in front-matter 'created'
description: "Copy of 04 September email sent to all NeSI users with current Nearline allocations"
status: new
search:

Check warning on line 5 in docs/General/Announcements/Preparing_to_move_data_to_NeSI_long_term_storage.md

View workflow job for this annotation

GitHub Actions / Check page meta

meta_unexpected_key

Unexpected parameter in front-matter 'search'
boost: 2
---

As part of NeSI's [shift to new platforms](https://www.nesi.org.nz/platform-refresh), our long-term storage service is changing. Our new platform will address diverse data management needs across various research fields, and deliver an easier-to-use and scaleable storage tool. We'll be sharing more details soon.

Check warning on line 9 in docs/General/Announcements/Preparing_to_move_data_to_NeSI_long_term_storage.md

View workflow job for this annotation

GitHub Actions / Check Spelling

spelling

Word 'scaleable' is misspelled.
Expand All @@ -22,7 +24,8 @@ Steps involved:

## Questions?

We have an [FAQ page](../../General/FAQs/Common_questions_about_the_platform_refresh/) to help answer common questions and
[weekly Online Office Hours](../..//Getting_Started/Getting_Help/Weekly_Online_Office_Hours/) with our support team, who are ready to answer or talk through any questions or issues you might have. You can also reach out anytime via email. {% include "partials/support_request.html" %}
We have an [FAQ page](../FAQs/Common_questions_about_the_platform_refresh/) to help answer common questions and
[weekly Online Office Hours](../../Getting_Started/Getting_Help/Weekly_Online_Office_Hours/) with our support team, who are ready to answer or talk through any questions or issues you might have. You can also reach out anytime via email.
{% include "partials/support_request.html" %}

We are ready to work with you ensure this transition is a smooth process and the new storage platform delivers an improved experience.
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,8 @@ tags:
- milan

Check warning on line 7 in docs/General/Announcements/Preparing_your_code_for_use_on_NeSIs_new_HPC_platform.md

View workflow job for this annotation

GitHub Actions / Check Spelling

spelling

Word 'milan' is misspelled.
description: Overview of some of the changes you need to be aware of when porting code from from Māui to Mahuika

Check warning on line 8 in docs/General/Announcements/Preparing_your_code_for_use_on_NeSIs_new_HPC_platform.md

View workflow job for this annotation

GitHub Actions / Check Prose

lexical_illusions.misc

'There's a lexical illusion here: a word is repeated.'
title: Preparing your code for use on NeSI's new HPC platform

Check warning on line 9 in docs/General/Announcements/Preparing_your_code_for_use_on_NeSIs_new_HPC_platform.md

View workflow job for this annotation

GitHub Actions / Check page meta

title_length

Title 'Preparing your code for use on NeSI's new HPC platform' is too long. Try to keep it under 24 characters to avoid word wrapping in the nav.
vote_count: 0
vote_sum: 0
zendesk_article_id: 8817840423439
zendesk_section_id: 200732737
search:

Check warning on line 10 in docs/General/Announcements/Preparing_your_code_for_use_on_NeSIs_new_HPC_platform.md

View workflow job for this annotation

GitHub Actions / Check page meta

meta_unexpected_key

Unexpected parameter in front-matter 'search'
boost: 0.1
boost: 2
---

## Background
Expand All @@ -25,13 +21,13 @@ and we anticipate migrating users in a staggered manner starting in July 2024. W
We anticipate teams might require assistance with getting ready, so we’re providing wrap-around support. This page provides an overview of how to familiarise yourself
with infrastructure similar to the new environment in advance. We explain the ways it will differ from Māui and Mahuika's Broadwell nodes, and actions you may need to take to prepare your project for migration.

Below is a quick overview of some of the changes you need to be aware of when porting code from from Māui to Mahuika:
Below is a quick overview of some of the changes you need to be aware of when porting code from Māui to Mahuika:

| Māui | Mahuika | comments |
|------------------------|-----------------------|-------------------------------------|
| NA | `module purge`
| `module avail -S X` | `module spider X` | search for module X |
| `module load PrgEnv-cray/6.0.10` | NA | no Cray compiler on Mahuika milan nodes |
| `module load PrgEnv-cray/6.0.10` | NA | no Cray compiler on Mahuika Milan nodes |
| `module load craype-hugepages*M` | NA | |
| `module load PrgEnv-intel` | `module load intel` | Intel MPI and Intel compilers |
| `module load PrgEnv-gnu` | `module load gimkl` | Intel MPI and GNU compilers |
Expand All @@ -44,7 +40,7 @@ Below is a quick overview of some of the changes you need to be aware of when po

## Test your code on Mahuika

The platform NeSI has selected to replace Mahuika is most similar to the
The platform NeSI has selected to replace Mahuika is most similar to the
[Mahuika AMD Milan compute nodes](../../General/Announcements/Mahuikas_new_Milan_CPU_nodes_open_to_all_NeSI_users.md) than nodes on other partitions.
So, we'll be using the Milan nodes to validate any issues, mitigating risks of your subsequent migration to the new platform.

Expand All @@ -62,13 +58,13 @@ early stage validation process is as quick and painless as possible.

The module command works much the same way on Mahuika, though it happens
to be a different implementation ("Lmod") with a few extra features.
 You will probably find its extra search command `module spider` to be
faster and more useful than the familiar `module avail`.  
You will probably find its extra search command `module spider` to be
faster and more useful than the familiar `module avail`.

If you currently use software on Māui that we have provided via
environment modules, then please check to see if we have it installed on
Mahuika (note that it is unlikely to be the same version) and let us
know about anything that you can't find.  If you compile your own
know about anything that you can't find. If you compile your own
software, then see below.

### Slurm options
Expand All @@ -77,31 +73,31 @@ software, then see below.

There are several partitions available to NeSI jobs on Mahuika, however
for the purposes of migrating from Māui and future-proofing, we
recommend the "milan" partition. As its name suggests, that partition
recommend the `milan` partition. As its name suggests, that partition

Check warning on line 76 in docs/General/Announcements/Preparing_your_code_for_use_on_NeSIs_new_HPC_platform.md

View workflow job for this annotation

GitHub Actions / Check Spelling

spelling

Word 'milan' is misspelled.
has AMD Milan (Zen3) CPUs, while the rest of Mahuika has Intel Broadwell
CPUs.

If for any reason you want to use any of the other Mahuika partitions,see
If for any reason you want to use any of the other Mahuika partitions,see
[Mahuika Slurm Partitions](../../Scientific_Computing/Running_Jobs_on_Maui_and_Mahuika/Mahuika_Slurm_Partitions.md) for
an overview and
an overview and
[Milan Compute Nodes](../../Scientific_Computing/Running_Jobs_on_Maui_and_Mahuika/Milan_Compute_Nodes.md) for
the differences between them and *milan*.

Check warning on line 84 in docs/General/Announcements/Preparing_your_code_for_use_on_NeSIs_new_HPC_platform.md

View workflow job for this annotation

GitHub Actions / Check Spelling

spelling

Word 'milan' is misspelled.

#### Shared nodes

Māui is scheduled by node while Mahuika is scheduled by core, so small
jobs can share Mahuika nodes, while on Māui nodes are exclusively
occupied by a single job at a time. 
occupied by a single job at a time.

When submitting an MPI job you have (at least) three options:

- Request a number of tasks without worrying what nodes they land on.
 That is OK for quick tests, but probably not optimal for real work
That is OK for quick tests, but probably not optimal for real work
as it both increases dependence on the interconnect and fragments
node resources, as such job submissions end up much more scattered
than they would on Māui.
- Request a number of tasks and a number (or range) of nodes.
- Request a number of nodes and a number of tasks per node.  This is
- Request a number of nodes and a number of tasks per node. This is
appropriate for most Māui-sized jobs, and by requesting all of the
CPUs on a node better isolates the job from contention with other
jobs over socket-level or node-level resources such as memory
Expand All @@ -112,19 +108,16 @@ When submitting an MPI job you have (at least) three options:
Since most ex-Māui jobs will want to take whole nodes, it is important
to be aware of the size of those nodes:

| | Māui | Mahuika (milan partition) |
| | Māui | Mahuika (`milan` partition) |

Check warning on line 111 in docs/General/Announcements/Preparing_your_code_for_use_on_NeSIs_new_HPC_platform.md

View workflow job for this annotation

GitHub Actions / Check Spelling

spelling

Word 'milan' is misspelled.
|-------|--------------|---------------------------|
|   | | |
| cores | 40 | 128 |
| CPUs | 80 | 256 |
| RAM | 90 or 180 GB | 460 or 960 GB |



### Temporary files

In Mahuika batch scripts, please replace any mention of `/tmp` with
`$TMPDIR`. 
`$TMPDIR`.

## Porting your software

Expand All @@ -134,7 +127,7 @@ If you have been compiling software on Māui you will be familiar with
the CPE, the "Cray Programming Environment" compiler wrappers (`ftn`,
`cc` and their underlying infrastructure) which allow you to switch
between the GCC, Intel, and Cray compilers while using the same command
lines.  CPE is not supported on Mahuika, and so it will be necessary to
lines. CPE is not supported on Mahuika, and so it will be necessary to
use a compiler directly, for example `gfortran` or `gcc`.

We have GCC and Intel compilers (but not the Cray compiler) available on
Expand Down Expand Up @@ -174,4 +167,3 @@ or pop in to one of our [weekly Online Office Hours](../../Getting_Started/Getti
to chat with Support staff one-to-one.

No question is too small - don't hesitate to reach out.

Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
---

Check warning on line 1 in docs/General/FAQs/Common_questions_about_the_platform_refresh.md

View workflow job for this annotation

GitHub Actions / Check page meta

meta.siblings

Parent category 'FAQs' has too many children (29). Try to keep number of items in a category under '8', maybe add some new categories?
description: Common questions researchers have about NeSI's platform refresh in 2024.
status: new
search:
boost: 2
---

NeSI is evolving its technology platform to ensure Aotearoa's national eResearch infrastructure and services are more accessible, responsive, and flexible to your needs. For more details [visit the NeSI website](https://www.nesi.org.nz/platform-refresh).

Check warning on line 8 in docs/General/FAQs/Common_questions_about_the_platform_refresh.md

View workflow job for this annotation

GitHub Actions / Check Spelling

spelling

Word 'Aotearoa's' is misspelled.
Expand Down Expand Up @@ -34,11 +37,10 @@ The platform NeSI has selected to replace Mahuika is similar to Mahuika's AMD Mi

Some projects on Māui will move to the new NeSI infrastructure. We have been in touch with those Māui projects and given them a small allocation on Mahuika which can be used to validate the software they need is available (or can be built) on Mahuika's AMD Milan nodes and works as expected. All members of the Māui project can use this Mahuika allocation. Visit this [how-to guide for instructions](https://docs.nesi.org.nz/General/Announcements/Preparing_your_code_for_use_on_NeSIs_new_HPC_platform/) of how to test your workloads on Milan.

## Does this affect Nearline?
## Does this affect Nearline?

All NeSI compute and storage services will leverage the new infrastructure. Nearline is changing, leveraging different technology to deliver an easier to use cold storage solution. We'll share more details on this soon. No action is required from you at this stage. If you have workflows which touch Nearline we will update our docs as the replacement service comes online.

## I have more questions that aren't covered here. Where can I go or who can I talk to for more information?

Reach out anytime - no question is too small. We are ready to respond - email us at [email protected] and we also invite you to join our [weekly Online Office Hours](https://docs.nesi.org.nz/Getting_Started/Getting_Help/Weekly_Online_Office_Hours/) to chat with us one-to-one.

Reach out anytime - no question is too small. We are ready to respond - email us at [[email protected]](mailto:[email protected]) and we also invite you to join our [weekly Online Office Hours](https://docs.nesi.org.nz/Getting_Started/Getting_Help/Weekly_Online_Office_Hours/) to chat with us one-to-one.
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,18 @@
created_at: 2024-11-27
description:
tags: [refresh]
status: new
search:
boost: 2
---

Migration of your data from GPFS to WEKA will be ongoing for several weeks.
We will be copying across a few project directories in parallel at a time, using _rsync_.
As each project directory is completed, the home directories of that project’s members will also be copied.
To keep the WEKA copy of your data as fresh as possible, we will continue cycling through the projects that way, repeatedly syncing your directories from GPFS to WEKA until you ask us to stop.

On the second and sucessive rounds of these synchronisations, the nobackup directories will also be examined. However **by default nothing will be migrated from nobackup directories**, as they often contain large amounts of data which is too transient to be worth preserving. You can override that, specifying nobackup content which should be preserved, as described below.
On the second and successive rounds of these synchronisations, the nobackup directories will also be examined.
However **by default nothing will be migrated from nobackup directories**, as they often contain large amounts of data which is too transient to be worth preserving. You can override that, specifying nobackup content which should be preserved, as described below.

Check warning on line 16 in docs/Storage/File_Systems_and_Quotas/Moving_to_the_new_filesystem.md

View workflow job for this annotation

GitHub Actions / Check Prose

consistency.spacing

'Inconsistent spacing after period (1 vs. 2 spaces).'

## Controlling which data gets migrated

Expand All @@ -31,11 +35,11 @@ Each line of the file (other than comment lines and blank lines) specifies a rul

A pattern can be as simple as the name of a subdirectory, but can also include wildcards:

- `*` matches anything other than a slash `/`, ie: any component of a path.
- `*` matches anything other than a slash `/`, i.e: any component of a path.

- `**` matches anything, so `keep**` would include all of the contents of a directory named `~/keepthis`.

- A trailing `dir_name/***` matches both `dir_name/` and `dir_name/**`, ie: the directory and all its contents.
- A trailing `dir_name/***` matches both `dir_name/` and `dir_name/**`, i.e: the directory and all its contents.

- A leading `/` anchors the pattern in the top directory, so `/tmp` matches `~/tmp` but not `~/other/tmp`.

Expand Down Expand Up @@ -74,7 +78,8 @@ For nobackup directories we use the same filter rules as above with one addition
- *
```

so you will have to override that if you want anything from your nobackup directory migrated into WEKA. eg:
so you will have to override that if you want anything from your nobackup directory migrated into WEKA.
e.g:

```sh
echo '+ *' > /nesi/nobackup/nesi99999/.RSYNC_FILTER
Expand Down

0 comments on commit 3062743

Please sign in to comment.