nativesdk-clang does not find its sanitizer/*.h include files

[dkl]

Hi,

Describe the bug
When trying to compile projects using #include <sanitizer/asan_interface.h> (for example libabsl) by using clang from the Yocto SDK with nativesdk-clang and compiler-rt-sanitizers, it shows fatal error: 'sanitizer/asan_interface.h' file not found. The file exists in $OECORE_TARGET_SYSROOT/usr/lib/clang/$version/include, but clang only uses $OECORE_NATIVE_SYSROOT/usr/lib/clang/$version/include and $OECORE_TARGET_SYSROOT/usr/include.

To Reproduce
Tested with poky + meta-clang, both branch scarthgap, with the following settings appended to local.conf:

CLANGSDK = "1"
TOOLCHAIN_TARGET_TASK:append = " compiler-rt-sanitizers"

and then

bitbake core-image-minimal -cpopulate_sdk

and then installing the SDK and using a compile command like $CLANGCC -fsanitize=address foo.c -c, where foo.c is simply

#include <sanitizer/asan_interface.h>

Expected behavior
With gcc and gcc-sanitizers it works fine, so I'm assuming that clang should automatically use the include path from compiler-rt-sanitizers too.

Additional context
Possible work-around: Add -isystem "$SDKTARGETSYSROOT/usr/lib/clang/18.1.5/include" when compiling affected projects

[FederAndInk]

I too have this issue for the native and target toolchain, clang is installed under /usr/lib/clang/$version but the sanitizers are under /usr/lib/clang/$version.$minor.$patch

[kraj]

@dkl and @FederAndInk can you test the patch in #994 and report back ?

[FederAndInk]

Sure, thanks, testing this right now 👍

[kraj]

Sure, thanks, testing this right now 👍

I think the problem is that the compiler headers path it pointing to recipe-sysroot-native and not recipe-sysroot. The patch I posted will not solve that problem. On the other hand if you add

TOOLCHAIN_TARGET_TASK:append = " nativesdk-compiler-rt-sanitizers"

This will work.

[FederAndInk]

Actually I am building clang for the target and using it on the "target", with the patch you posted, it is now working

[FederAndInk]

I will be testing the sdk variant soon

[FederAndInk]

TOOLCHAIN_TARGET_TASK:append = " nativesdk-compiler-rt-sanitizers"

This will work.

isn't a bit weird to put a nativesdk version in TOOLCHAIN_TARGET_TASK? What will that do? will the target libraries be generated and available to be linked against?

[FederAndInk]

ok so looks like you're right, trying ${CLANGCXX} test.cpp -v we can see clang search paths looking in target sysroot but 1 (usr/lib/clang/18) searching for it in the native sysroot:

Documents/sdk/sysroots/corei7-64-oe-linux/lib/x86_64-oe-linux/13.3.0/../../../include/c++/13.3.0                            
Documents/sdk/sysroots/corei7-64-oe-linux/lib/x86_64-oe-linux/13.3.0/../../../include/c++/13.3.0/x86_64-oe-linux            
Documents/sdk/sysroots/corei7-64-oe-linux/lib/x86_64-oe-linux/13.3.0/../../../include/c++/13.3.0/backward                   
Documents/sdk/sysroots/x86_64-vendor-linux/usr/lib/clang/18/include                                                           
Documents/sdk/sysroots/corei7-64-oe-linux/usr/include

[kraj]

TOOLCHAIN_TARGET_TASK:append = " nativesdk-compiler-rt-sanitizers"
This will work.

isn't a bit weird to put a nativesdk version in TOOLCHAIN_TARGET_TASK? What will that do? will the target libraries be generated and available to be linked against?

yeah I dont think its the right solution, I was just saying this to prove the problem as we understand. I think correct fix would be to fix the compiler to look into target sysroot for compiler headers.

[kraj]

Actually I am building clang for the target and using it on the "target", with the patch you posted, it is now working

OK, so this patch atleast fixes one case. So maybe we can merge this one and keep searching to solve the second one in nativesdk cross compiler include search path fixes.

[FederAndInk]

TOOLCHAIN_TARGET_TASK:append = " nativesdk-compiler-rt-sanitizers"
This will work.

isn't a bit weird to put a nativesdk version in TOOLCHAIN_TARGET_TASK? What will that do? will the target libraries be generated and available to be linked against?

yeah I dont think its the right solution, I was just saying this to prove the problem as we understand. I think correct fix would be to fix the compiler to look into target sysroot for compiler headers.

ok I see your point

Actually I am building clang for the target and using it on the "target", with the patch you posted, it is now working

OK, so this patch atleast fixes one case. So maybe we can merge this one and keep searching to solve the second one in nativesdk cross compiler include search path fixes.

right, thanks again :)
btw, I see that debian is doing a similar thing they install the toolchain under usr/lib/llvm-<major> and make usr/lib/clang/<major> and usr/lib/clang/<major>.<minor>.<patch> symlinks to ../llvm-<major>

[FederAndInk]

Hey, thanks for merging a first part of the solution :)
Can we hope for a backport to the scarthgap branch?

Also, I think this issue should be re-openned as #994 only fixes the on-target part, what do you think @kraj?

[kraj]

Yes reopen it and please sent a pull request for scarthgap

[dkl]

After the patch which adds the symlink to compiler-rt-sanitizers, there is a conflict with compiler-rt-dev which provides a single file /usr/lib/clang/18.1.8/include/orc/c_api.h. From dnf:

  file /usr/lib/clang/18.1.8 conflicts between attempted installs of compiler-rt-sanitizers-18.1.8-r0.core2_64 and compiler-rt-dev-18.1.8-r0.core2_64

(/usr/lib/clang/18.1.8 is a symlink in one, but a normal directory in the other)

[FederAndInk]

right, I guess the same logic should be applied to compiler-rt recipe too