Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test availability of -march=native with clang #105

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

JoachimSchurig
Copy link

Description

-march=native is not supported by clang on all platforms, particularly not on ARM platforms. Using Ubuntu 20.04 on AWS ARM machines e.g. does not support it. browsermt/marian-dev however sets it. The resulting error message is misleading, telling that no thread library can be found.
The displayed error comes from a compile test in a submodule that uses the modified compiler options including -march=native, which fails, hence gets analysed as no pthread support. But even when making that test pass manually, all compile attempts of course fail subsequently.

See a one line test program:
$ echo "int main() { return 0; }" > test.c
$ clang -march=native test.c -o test
clang: error: the clang compiler does not support '-march=native'

$ clang --version
clang version 10.0.0-4ubuntu1
Target: aarch64-unknown-linux-gnu

List of changes:

  • adding a test in the main CMakeLists.txt to test compile with -march=native if clang, and replacing with -mtune=native -mcpu=native if not successful

Added dependencies: none

How to test

Install e.g. Ubuntu 20.04 on an AWS ARM instance, clone marian-dev, create a build directory and cd into it, then run
CC=clang CXX=clang++ cmake -DUSE_SENTENCEPIECE:BOOL=ON -DCOMPILE_CPU:BOOL=ON -DCOMPILE_CUDA:BOOL=OFF -DUSE_CUDNN:BOOL=OFF -DUSE_FBGEMM:BOOL=OFF -DUSE_INTGEMM:BOOL=OFF -DUSE_NCCL:BOOL=OFF -DUSE_STATIC_LIBS:BOOL=ON -DSPM_ENABLE_SHARED:BOOL=OFF <path_to_sources>

This will fail on an unpatched codebase with supposedly the thread lib not found. But when you check the CMakeFiles/CMakeError.log file, you will see that the problem is the -march=native in the test compile for pthread.

After applying the patch, the cmake setup and a subsequent compile pass.

This has been tested on: AWS ARM with Ubuntu 20.04, AWS x86 with Ubuntu 20.04 (where clang passed already before the patch), and MacOS with Apple M1 (where clang passed already before the patch). gcc has not been tested on MacOS, but on all other platforms, and passes.

Checklist

  • I have tested the code manually
  • I have run regression tests
  • I have read and followed CONTRIBUTING.md
  • I have updated CHANGELOG.md

JoachimSchurig and others added 2 commits October 5, 2023 16:25
clang does not support -march=native on all platforms. Therefore do a test compile if -march=native is requested and the compiler is set to clang. 
If the test fails, replace -march=native with -mtune=native -mcpu=native (which should be supported everywhere)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant