Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ggml: fix arm build with gcc #10895

Merged
merged 1 commit into from
Dec 19, 2024
Merged

Conversation

angt
Copy link
Contributor

@angt angt commented Dec 19, 2024

So, we continue :)

GCC doesn't like -mcpu=native+ext (and not -march=native+ext either):

$ gcc -mcpu=native+dotprod -o dotprod dotprod.c
cc1: error: unknown value ‘native+dotprod’ for ‘-mcpu’
cc1: note: valid arguments are: cortex-a34 cortex-a35 cortex-a53 cortex-a57 ...

The new detection mechanism doesn't work on my graviton4 (but luckily -mcpu=native works):

-- ARM detected
-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- Performing Test GGML_COMPILER_SUPPORT_DOTPROD
-- Performing Test GGML_COMPILER_SUPPORT_DOTPROD - Failed
-- Performing Test GGML_COMPILER_SUPPORT_I8MM
-- Performing Test GGML_COMPILER_SUPPORT_I8MM - Failed
-- ARM feature DOTPROD enabled
-- ARM feature SVE enabled
-- ARM feature MATMUL_INT8 enabled
-- ARM feature FMA enabled
-- ARM feature FP16_VECTOR_ARITHMETIC enabled
-- Adding CPU backend variant ggml-cpu: -mcpu=native 

This PR propose to use -mcpu=native to detect the CPU like this:

gcc -mcpu=native -E -v - </dev/null 2>&1 | grep -oE "\-mcpu=[^ ']+" -m 1
-mcpu=neoverse-v2+crc+sve2-aes+sve2-sha3

With this PR we get:

GCC:

-- ARM detected
-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- Performing Test GGML_COMPILER_SUPPORT_DOTPROD
-- Performing Test GGML_COMPILER_SUPPORT_DOTPROD - Success
-- Performing Test GGML_COMPILER_SUPPORT_I8MM
-- Performing Test GGML_COMPILER_SUPPORT_I8MM - Success
-- ARM feature DOTPROD enabled
-- ARM feature SVE enabled
-- ARM feature MATMUL_INT8 enabled
-- ARM feature FMA enabled
-- ARM feature FP16_VECTOR_ARITHMETIC enabled
-- Adding CPU backend variant ggml-cpu: -mcpu=neoverse-v2+crc+sve2-aes+sve2-sha3+dotprod+i8mm 

clang:

-- ARM detected
-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- ARM -mcpu not found, -mcpu=native will be used
-- Performing Test GGML_COMPILER_SUPPORT_DOTPROD
-- Performing Test GGML_COMPILER_SUPPORT_DOTPROD - Success
-- Performing Test GGML_COMPILER_SUPPORT_I8MM
-- Performing Test GGML_COMPILER_SUPPORT_I8MM - Success
-- ARM feature DOTPROD enabled
-- ARM feature SVE enabled
-- ARM feature MATMUL_INT8 enabled
-- ARM feature FMA enabled
-- ARM feature FP16_VECTOR_ARITHMETIC enabled
-- Adding CPU backend variant ggml-cpu: -mcpu=native+dotprod+i8mm

Where -mcpu=native+ext works...

Signed-off-by: Adrien Gallouët <[email protected]>
@github-actions github-actions bot added the ggml changes relating to the ggml tensor library for machine learning label Dec 19, 2024
@angt
Copy link
Contributor Author

angt commented Dec 19, 2024

I was tricked by my mac ^^

$ gcc --version
Apple clang version 16.0.0 (clang-1600.0.26.6)
Target: arm64-apple-darwin24.1.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

@slaren
Copy link
Collaborator

slaren commented Dec 19, 2024

I tried testing on M3 Max with gcc from brew, but it does not seem to work very well, it fails during the assembler with "error: instruction requires: i8mm". So I guess no gcc on Apple.

@slaren slaren merged commit a3c33b1 into ggerganov:master Dec 19, 2024
48 checks passed
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ggml changes relating to the ggml tensor library for machine learning
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants