Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support in archdetect for detecting A64FX #608

Merged
merged 7 commits into from
Jan 10, 2025
Merged

add support in archdetect for detecting A64FX #608

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
cb1672b
add support for detecting A64FX to archdetect
boegel Jun 13, 2024
f376b9a
Merge branch '2023.06-software.eessi.io' into 2023.06-software.eessi.…
boegel Jan 10, 2025
5bad2d9
use 0x46 as 'CPU implementer' field for A64FX
boegel Jan 10, 2025
e53ac16
fall back to 'CPU implementer' field from /proc/cpuinfo if 'Vendor ID…
boegel Jan 10, 2025
f112406
add Deucalion-Rocky85.all.output for archdetect tests
boegel Jan 10, 2025
d82a188
set correct 'CPU implementer' for graviton2/graviton3
boegel Jan 10, 2025
562fd4d
fix logic for determining field to use to determine CPU flags
boegel Jan 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/tests_archdetect.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ jobs:
- x86_64/amd/zen3/Azure-CentOS7-7V73X
- x86_64/amd/zen4/Azure-Alma8-9V33X
- x86_64/amd/zen4/Shinx-RHEL8-9654
- aarch64/a64fx/Deucalion-Rocky85
- aarch64/neoverse_n1/Azure-Ubuntu20-Altra
- aarch64/neoverse_n1/AWS-awslinux-graviton2
- aarch64/neoverse_v1/AWS-awslinux-graviton3
Expand Down
9 changes: 6 additions & 3 deletions init/arch_specs/eessi_arch_arm.spec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# ARM CPU architecture specifications (see https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html for guidance)
# Software path in EESSI | Vendor ID | List of defining CPU features
# CPU implementers: 0x41 (ARM), 0x46 (Fujitsu) - also see https://github.com/hrw/arm-socs-table/blob/main/data/socs.yml

# Software path in EESSI | 'Vendor ID' or 'CPU implementer' | List of defining CPU features
"aarch64/a64fx" "0x46" "asimdhp sve" # Fujitsu A64FX
"aarch64/neoverse_n1" "ARM" "asimddp" # Ampere Altra
"aarch64/neoverse_n1" "" "asimddp" # AWS Graviton2
"aarch64/neoverse_n1" "0x41" "asimddp" # AWS Graviton2
"aarch64/neoverse_v1" "ARM" "asimddp svei8mm"
"aarch64/neoverse_v1" "" "asimddp svei8mm" # AWS Graviton3
"aarch64/neoverse_v1" "0x41" "asimddp svei8mm" # AWS Graviton3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when there are multiple vendors implementing the instruction set, like with Gravition/Grace for "aarch64/neoverse_v2"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is only necessary when there is a doubt about what the match is

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html only lists Graviton 3 as CPU supporting Neoverse V1, so that makes it very unlikely in practice we'll run into something that's detected as Neoverse V1 but isn't a Graviton 3 (at least today).

For neoverse_v2 with Graviton 4 vs Grace, there's a more complex situation though, because Graviton 4 supports CPU instructions like paca, pacg, rng which Grace doesn't, and the other way around (like sm3, sm4, svesm4).
It gets even more interesting when Google Axion is taken into account, since that doesn't support sbss, which the other two do...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nevermind, it is right there in the file, just means we need to add each supported CPU explicitly. That seems like a bit of a pity, doesn't it make the archdetect selection very conservative if it encounters a CPU it hasn't seen before? It doesn't matter if the CPU has all the required features

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For neoverse_v2 with Graviton 4 vs Grace, there's a more complex situation though, because Graviton 4 supports CPU instructions like paca, pacg, rng which Grace doesn't, and the other way around (like sm3, sm4, svesm4). It gets even more interesting when Google Axion is taken into account, since that doesn't support sbss, which the other two do...

Isn't that quite a big problem? Can we disable some instructions? I guess compilers would be caught by surprise...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This data is probably pretty useful for us: https://github.com/hrw/arm-socs-table/tree/main/cpuinfo-data

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we shouldn't have a much more considered set of CPU features that reflect the CPU where the software was built for the target and then match that? This would do a much better job of future matching CPUs. Is it really the case that CPU features alone are not enough to distinguish A64FX from Neoverse_N1?

Not going to hold this PR back for that discussion, will open an issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also wondering if we can instruct the compiler to not emit particular instructions even if they're supported.
My gut says that should be possible, but I'm not sure it actually is...

Let's follow up on that in a dedicated issue though => #845

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For A64FX vs Neoverse N1/V1, see the /proc/cpuinfo dumps we have under tests/archdetect/.

23 changes: 17 additions & 6 deletions init/eessi_archdetect.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,16 +111,27 @@ cpupath(){
update_arch_specs "$base_dir/arch_specs/${spec_file}"

# Identify the host CPU vendor
local cpu_vendor_tag="vendor[ _]id"
local cpu_vendor=$(get_cpuinfo "$cpu_vendor_tag")
local cpu_vendor=$(get_cpuinfo "vendor[ _]id")
if [ "${cpu_vendor}" == "" ]; then
cpu_vendor=$(get_cpuinfo "cpu[ _]implementer")
fi
log "DEBUG" "cpupath: CPU vendor of host system: '$cpu_vendor'"

# Identify the host CPU flags or features
local cpu_flag_tag='flags'
# cpuinfo systems print different line identifiers, eg features, instead of flags
[ "${cpu_vendor}" == "ARM" ] && cpu_flag_tag='flags'
[ "${machine_type}" == "aarch64" ] && [ "${cpu_vendor}x" == "x" ] && cpu_flag_tag='features'
[ "${machine_type}" == "ppc64le" ] && cpu_flag_tag='cpu'
local cpu_flag_tag;
if [ "${cpu_vendor}" == "ARM" ]; then
# if CPU vendor field is ARM, then we should be able to determine CPU microarchitecture based on 'flags' field
cpu_flag_tag='flags'
# if 64-bit Arm CPU without "ARM" as vendor ID, we need to take into account 'features' field
elif [ "${machine_type}" == "aarch64" ]; then
cpu_flag_tag='features'
# on 64-bit POWER, we need to look at 'cpu' field
elif [ "${machine_type}" == "ppc64le" ]; then
cpu_flag_tag='cpu'
else
cpu_flag_tag='flags'
fi

local cpu_flags=$(get_cpuinfo "$cpu_flag_tag")
log "DEBUG" "cpupath: CPU flags of host system: '$cpu_flags'"
Expand Down
1 change: 1 addition & 0 deletions tests/archdetect/aarch64/a64fx/Deucalion-Rocky85.all.output
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
aarch64/a64fx:aarch64/generic
8 changes: 8 additions & 0 deletions tests/archdetect/aarch64/a64fx/Deucalion-Rocky85.cpuinfo
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
processor : 0
BogoMIPS : 200.00
Features : fp asimd evtstrm sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm fcma dcpop sve
CPU implementer : 0x46
CPU architecture: 8
CPU variant : 0x1
CPU part : 0x001
CPU revision : 0
1 change: 1 addition & 0 deletions tests/archdetect/aarch64/a64fx/Deucalion-Rocky85.output
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
aarch64/a64fx
Loading