[BUG] Compile modern_bpf error #1719

yzewei · 2024-03-01T09:40:30Z

Describe the bug
make drivers_test

/home/yzw/rd/libs/driver/modern_bpf/helpers/extract/extract_from_kernel.h:648:49: error: member reference base type 'struct percpu_counter[4]' is not a structure or union
  648 |                 BPF_CORE_READ_INTO(&swap_entries, mm, rss_stat.count[MM_SWAPENTS].counter);
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~

The code files involved include :
modern_bpf/programs/tail_called/events/syscall_dispatched_events/execve.bpf.c modern_bpf/programs/tail_called/events/syscall_dispatched_events/execveat.bpf.c modern_bpf/helpers/extract/extract_from_kernel.h

How to reproduce it
First judge the kernel version, and then execute the corresponding operation. like this:

#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 3, 0)

instead of

if(bpf_core_field_exists(((struct kernel_cap_struct *)0)->cap)){...}

Expected behaviour

Screenshots

Environment

Falco version:

System info:

Cloud provider or hardware configuration:
OS:

Kernel:

Installation method:

Additional context

Tasks

Give feedback

feat(driver): Add loongarch64 kernel module support #1654

area/driver-kmod dco-signoff: yes do-not-merge/hold release-note-none size/XXL
Options

Tasks

Give feedback

add ebpf support for loongarch64
Options

The text was updated successfully, but these errors were encountered:

Andreagit97 · 2024-03-05T10:49:47Z

Thank you for reporting, can you provide more info?

Which kernel version are you running?
Which architecture are you using?
Can you generate the vmlinux for your system with bpftool bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h ?

yzewei · 2024-03-05T11:18:58Z

Thank you for reporting, can you provide more info?

Which kernel version are you running?

Which architecture are you using?

Can you generate the vmlinux for your system with bpftool bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h ?

Thank you for your attention!
The kernel version we are currently using is:

6.8.0-rc5
The architecture is loongarch64, which is currently being adapted, including kmod, bpf, and modern-bpf modules
I can get vmlinux.h

Andreagit97 · 2024-03-06T10:53:11Z

Uhm ok looking at this compiler error

/home/yzw/rd/libs/driver/modern_bpf/helpers/extract/extract_from_kernel.h:648:49: error: member reference base type 'struct percpu_counter[4]' is not a structure or union
  648 |                 BPF_CORE_READ_INTO(&swap_entries, mm, rss_stat.count[MM_SWAPENTS].counter);
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~

it seems that even if your kernel supports the new rss_stat format (so struct percpu_counter instead of struct mm_rss_stat) the struct mm_rss_stat is not removed in your kernel and for this reason, we enter in the wrong branch... we should enter in the else but according to the compiler error we are falling in the first one...

	if(bpf_core_type_exists(struct mm_rss_stat))
	{
		BPF_CORE_READ_INTO(&swap_entries, mm, rss_stat.count[MM_SWAPENTS].counter);
	}
	else
	{
		struct mm_struct___v6_2 *mm_v6_2 = (void *)mm;
		BPF_CORE_READ_INTO(&swap_entries, mm_v6_2, rss_stat[MM_SWAPENTS].count);
	}

If you run these 2 commands on your machine, they should both return true, can you check it?

bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "mm_rss_stat" && echo "true" || echo "false" 
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "percpu_counter rss_stat" && echo "true" || echo "false"

yzewei · 2024-03-07T01:17:06Z

Uhm ok looking at this compiler error
/home/yzw/rd/libs/driver/modern_bpf/helpers/extract/extract_from_kernel.h:648:49: error: member reference base type 'struct percpu_counter[4]' is not a structure or union
  648 |                 BPF_CORE_READ_INTO(&swap_entries, mm, rss_stat.count[MM_SWAPENTS].counter);
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
it seems that even if your kernel supports the new rss_stat format (so struct percpu_counter instead of struct mm_rss_stat) the struct mm_rss_stat is not removed in your kernel and for this reason, we enter in the wrong branch... we should enter in the else but according to the compiler error we are falling in the first one...
	if(bpf_core_type_exists(struct mm_rss_stat))
	{
		BPF_CORE_READ_INTO(&swap_entries, mm, rss_stat.count[MM_SWAPENTS].counter);
	}
	else
	{
		struct mm_struct___v6_2 *mm_v6_2 = (void *)mm;
		BPF_CORE_READ_INTO(&swap_entries, mm_v6_2, rss_stat[MM_SWAPENTS].count);
	}
If you run these 2 commands on your machine, they should both return true, can you check it?
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "mm_rss_stat" && echo "true" || echo "false" 
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "percpu_counter rss_stat" && echo "true" || echo "false" 

@Andreagit97 Certainly!
According to the command, the result can be obtained as follows:

[yzw@fedora libs]$ bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "mm_rss_stat" && echo "true" || echo "false" 
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "percpu_counter rss_stat" && echo "true" || echo "false" 
false
true

It's true that I didn't enter the else but entered the first branch. I'm trying to clear it. This is why...
In the vmlinux.h file there is really only struct percpu_counter rss_stat [4];
There was no mm_rss_stat

[yzw@fedora libs]$ find -name "vmlinux.h" | grep -rn "mm_rss_stat"
driver/modern_bpf/definitions/ppc64le/vmlinux.h:2125:struct mm_rss_stat {
driver/modern_bpf/definitions/ppc64le/vmlinux.h:2191:		struct mm_rss_stat rss_stat;
driver/modern_bpf/definitions/x86_64/vmlinux.h:1754:struct mm_rss_stat {
driver/modern_bpf/definitions/x86_64/vmlinux.h:1847:		struct mm_rss_stat rss_stat;
driver/modern_bpf/definitions/aarch64/vmlinux.h:2355:struct mm_rss_stat {
driver/modern_bpf/definitions/aarch64/vmlinux.h:2569:		struct mm_rss_stat rss_stat;
driver/modern_bpf/definitions/s390x/vmlinux.h:1668:struct mm_rss_stat {
driver/modern_bpf/definitions/s390x/vmlinux.h:1921:		struct mm_rss_stat rss_stat;
driver/modern_bpf/helpers/extract/extract_from_kernel.h:622:	 * `struct mm_rss_stat` doesn't exist anymore.
driver/modern_bpf/helpers/extract/extract_from_kernel.h:624:	if(bpf_core_type_exists(struct mm_rss_stat))

[yzw@fedora loongarch64]$ find -name "vmlinux.h" | grep -rn "rss_stat"
vmlinux.h:1558:		struct percpu_counter rss_stat[4];
vmlinux.h:99637:struct trace_event_raw_rss_stat {
vmlinux.h:99668:struct trace_event_data_offsets_rss_stat {};
vmlinux.h:99690:typedef void (*btf_trace_rss_stat)(void *, struct mm_struct *, int);

yzewei · 2024-03-07T06:28:38Z

I found that the reason is that the path to percpu_counter was not imported at compile time
It exists in the compiled kernel folder, but when I install it using make headers_install it is not installed at the path/usr/include/linux.
And when I specify the path to percpu_counter as C_INCLUDE_PATH, there is a conflict at compiling zlib

[  1%] Performing build step for 'zlib'
In file included from ./zconf.h:450,
                 from ./zlib.h:34,
                 from test/example.c:8:
/usr/include/sys/types.h:42:18: 错误：conflicting types for ‘loff_t’; have ‘__loff_t’ {或称 ‘long int’}
   42 | typedef __loff_t loff_t;
      |                  ^~~~~~
In file included from /home/yzw/workspace/linux-chenhuacai/include/linux/limits.h:6,
                 from /usr/include/bits/local_lim.h:38,
                 from /usr/include/bits/posix1_lim.h:161,
                 from /usr/include/limits.h:195,
                 from /usr/lib/gcc/loongarch64-redhat-linux/13/include/limits.h:205,
                 from /usr/lib/gcc/loongarch64-redhat-linux/13/include/syslimits.h:7,
                 from /usr/lib/gcc/loongarch64-redhat-linux/13/include/limits.h:34,
                 from ./zconf.h:424:
/home/yzw/workspace/linux-chenhuacai/include/linux/types.h:52:33: 附注：previous declaration of ‘loff_t’ with type ‘loff_t’ {或称 ‘long long int’}
   52 | typedef __kernel_loff_t         loff_t;
      |                                 ^~~~~~
/usr/include/sys/types.h:59:17: 错误：conflicting types for ‘dev_t’; have ‘__dev_t’ {或称 ‘long unsigned int’}
   59 | typedef __dev_t dev_t;
      |                 ^~~~~
/home/yzw/workspace/linux-chenhuacai/include/linux/types.h:21:33: 附注：previous declaration of ‘dev_t’ with type ‘dev_t’ {或称 ‘unsigned int’}
   21 | typedef __kernel_dev_t          dev_t;
      |                                 ^~~~~
In file included from /usr/include/sys/types.h:130:
/usr/include/bits/types/timer_t.h:7:19: 错误：conflicting types for ‘timer_t’; have ‘__timer_t’ {或称 ‘void *’}
    7 | typedef __timer_t timer_t;
      |                   ^~~~~~~
/home/yzw/workspace/linux-chenhuacai/include/linux/types.h:31:33: 附注：previous declaration of ‘timer_t’ with type ‘timer_t’ {或称 ‘int’}
   31 | typedef __kernel_timer_t        timer_t;
      |                                 ^~~~~~~
In file included from /usr/include/sys/types.h:155:
/usr/include/bits/stdint-intn.h:27:19: 错误：conflicting types for ‘int64_t’; have ‘__int64_t’ {或称 ‘long int’}
   27 | typedef __int64_t int64_t;

What should I do about it?

Andreagit97 · 2024-03-07T10:54:31Z

Sorry, I didn't get it... if mm_rss_stat is not defined in your kernel, how is it possible that you enter the if statement?

I found that the reason is that the path to percpu_counter was not imported at compile time

I don't understand how this is related to the fact the the code finds a definition of mm_rss_stat 🤔
Could you provide the dump of your local vmlinux.h (bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h)? I don't understand who is defining this mm_rss_stat ...

yzewei · 2024-03-19T12:41:49Z

@Andreagit97
Could this be due to the Loongarch architecture being different from other architectures? Is there any way I can debug bpf_core_type_exists?
Since I'm not very familiar with ebpf, I can't read the progress bar down here...

Andreagit97 · 2024-03-20T08:16:54Z

Could this be due to the Loongarch architecture being different from other architectures? Is there any way I can debug bpf_core_type_exists?

Uhm i'm not sure this is an issue with bpf_core_type_exists, in the end, this is a simple libbpf macro that calls a compiler built-in

/*
 * Convenience macro to check that provided named type
 * (struct/union/enum/typedef) exists in a target kernel.
 * Returns:
 *    1, if such type is present in target kernel's BTF;
 *    0, if no matching type is found.
 */
#define bpf_core_type_exists(type)					    \
	__builtin_preserve_type_info(*(typeof(type) *)0, BPF_TYPE_EXISTS)

The best way to debug is to obtain the vmlinux.h of your kernel and see if this type is here or not.
I've seen in our previous conversation that you were able to type these commands

bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "mm_rss_stat" && echo "true" || echo "false" 
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "percpu_counter rss_stat" && echo "true" || echo "false"

Can you just run this command and provide us with the vmlinux.h?

bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h

yzewei · 2024-03-20T08:17:42Z

The root cause of the problem is the problem of the clang built-in function under the loongarch architecture

yzewei · 2024-03-20T08:18:55Z

Could this be due to the Loongarch architecture being different from other architectures? Is there any way I can debug bpf_core_type_exists?

Uhm i'm not sure this is an issue with bpf_core_type_exists, in the end, this is a simple macro that calls a compiler built-in
/*
 * Convenience macro to check that provided named type
 * (struct/union/enum/typedef) exists in a target kernel.
 * Returns:
 *    1, if such type is present in target kernel's BTF;
 *    0, if no matching type is found.
 */
#define bpf_core_type_exists(type)					    \
	__builtin_preserve_type_info(*(typeof(type) *)0, BPF_TYPE_EXISTS)
The best way to debug is to obtain the vmlinux.h of your kernel and see if this type is here or not. I've seen in our previous conversation that you were able to type these commands
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "mm_rss_stat" && echo "true" || echo "false" 
bpftool btf dump file /sys/kernel/btf/vmlinux format c | grep -q "percpu_counter rss_stat" && echo "true" || echo "false" 
Can you just run this command and provide us with the vmlinux.h?
bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h

of course，https://github.com/libbpf/libbpf-bootstrap/blob/master/vmlinux/loongarch/vmlinux_602.h

yzewei · 2024-03-20T08:22:50Z

@Andreagit97 If you would like my vmlinux.h from TCE-metal, I would be happy to provide it, please visit
https://github.com/Loongson-Cloud-Community/Loongson-Cloud-Community/blob/main/docs/%E5%85%B6%E4%BB%96%E6%96%87%E6%A1%A3/vmlinux.h

os:fedora
kernel-version:6.8.0-rc5
llvm-version:18
clang-version:18
BTF:on
CO-RE:on

Regarding this issue, I am consulting the relevant personnel in the llvm community responsible for the loongarch architecture.
If you have any suggestions, you can contact me at any time, thank you very much!
Although I don't know why the built-in function has such a problem, I think you can try loongarch's vmlinux.h replication problem under x86 architecture

Andreagit97 · 2024-03-20T08:49:20Z

uhm ok i see... not sure this is the exact vmlinux of you machine but yes, in this vmlinux you provided there is no the mm_rss_stat struct, so yes this is strange... I've seen that you are using an high clang version (18), have you ever tried with a lower version? something like clang-14/clang-15? Locally i use clang-14, maybe something is changed

yzewei · 2024-03-20T08:57:15Z

uhm ok i see... not sure this is the exact vmlinux of you machine but yes, in this vmlinux you provided there is no the mm_rss_stat struct, so yes this is strange... I've seen that you are using an high clang version (18), have you ever tried with a lower version? something like clang-14/clang-15? Locally i use clang-14, maybe something is changed

Sir, I have provided the loongarch architecture vmlinux.h header file given by the libbpf official and the vmlinux.h generated by my machine respectively. Neither of these two header files from different sources mm_rss_stat structure

yzewei · 2024-03-20T09:01:30Z

use clang-14 error：

error: unable to create target: 'No available targets are compatible with triple "bpf"'

Andreagit97 · 2024-03-20T09:08:23Z

ok, I read again the issue, and we are missing something... you are compiling this on your machine so what is happening is perfectly legit, the compiler is just saying that it cannot find a definition for struct mm_rss_stat in your vmlinux.h, so we need to add it between missing definitions, sorry for that, I was convinced for some reason that you were executing the probe against your local machine...

yzewei · 2024-03-20T09:11:14Z

ok, I read again the issue, and we are missing something... you are compiling this on your machine so what is happening is perfectly legit, the compiler is just saying that it cannot find a definition for struct mm_rss_stat in your vmlinux.h, so we need to add it between missing definitions, sorry for that, I was convinced for some reason that you were executing the probe against your local machine...

Regarding this issue, if we wait for the built-in functions of clang to return to normal, it will take a long time. What are your suggestions?
Add arch detection to avoid this problem?
In #1654, we discussed the feasibility of switching to kernel version checking

Andreagit97 · 2024-03-20T09:34:43Z

Uhm let me explain better what is happening here. On your local machine, to compile the modern ebpf probe, you have probably added another vmlinux.h under this folder https://github.com/falcosecurity/libs/tree/0.15.0/driver/modern_bpf/definitions. In this vmlinux.h there is no definition for struct mm_rss_stat, while if you check in all the others vmlinux.h we have in this folder you will see that the struct mm_rss_stat is defined. This is because all the existing vmlinux.h are generated from a kernel version <= 6.2.

Now how can you solve this missing definition?
You need to add the definition by yourself in our struct_flavors.h file. So you need to add something like

#if defined(__TARGET_ARCH_loongarch)
struct mm_rss_stat {
	atomic_long_t count[4];
};
#endif

this should solve this compilation issue, but there could be others since your vmlinux.h is generated on a recent loongarch machine...

The best solution here would be to generate a vmlinux.h for loongarch in an older machine something with a kernel version <= 6.0.0, but not sure if you have a machine to do that

yzewei · 2024-03-21T00:56:20Z

Uhm let me explain better what is happening here. On your local machine, to compile the modern ebpf probe, you have probably added another vmlinux.h under this folder https://github.com/falcosecurity/libs/tree/0.15.0/driver/modern_bpf/definitions. In this vmlinux.h there is no definition for struct mm_rss_stat, while if you check in all the others vmlinux.h we have in this folder you will see that the struct mm_rss_stat is defined. This is because all the existing vmlinux.h are generated from a kernel version <= 6.2.

Now how can you solve this missing definition? You need to add the definition by yourself in our struct_flavors.h file. So you need to add something like
#if defined(__TARGET_ARCH_loongarch)
struct mm_rss_stat {
	atomic_long_t count[4];
};
#endif
this should solve this compilation issue, but there could be others since your vmlinux.h is generated on a recent loongarch machine...

The best solution here would be to generate a vmlinux.h for loongarch in an older machine something with a kernel version <= 6.0.0, but not sure if you have a machine to do that

mm_rss_stat structure does not exist when kernel version > 6.2.0, this definition may cause problems with project compilation.
I will try to compile a lower version of the kernel

yzewei · 2024-03-21T06:11:31Z

I wonder if we could change the criteria from mm_rss_stat to kernel version?
like this:
from

        if(bpf_core_type_exists(struct mm_rss_stat))
        {
                BPF_CORE_READ_INTO(&file_pages, mm, rss_stat.count[MM_FILEPAGES].counter);
                BPF_CORE_READ_INTO(&anon_pages, mm, rss_stat.count[MM_ANONPAGES].counter);
                BPF_CORE_READ_INTO(&shmem_pages, mm, rss_stat.count[MM_SHMEMPAGES].counter);
        }
        else
        {
                struct mm_struct___v6_2 *mm_v6_2 = (void *)mm;
                BPF_CORE_READ_INTO(&file_pages, mm_v6_2, rss_stat[MM_FILEPAGES].count);
                BPF_CORE_READ_INTO(&anon_pages, mm_v6_2, rss_stat[MM_ANONPAGES].count);
                BPF_CORE_READ_INTO(&shmem_pages, mm_v6_2, rss_stat[MM_SHMEMPAGES].count);
        }

to

#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 2, 0)
                BPF_CORE_READ_INTO(&file_pages, mm, rss_stat.count[MM_FILEPAGES].counter);
                BPF_CORE_READ_INTO(&anon_pages, mm, rss_stat.count[MM_ANONPAGES].counter);
                BPF_CORE_READ_INTO(&shmem_pages, mm, rss_stat.count[MM_SHMEMPAGES].counter);
#else
                struct mm_struct___v6_2 *mm_v6_2 = (void *)mm;
                BPF_CORE_READ_INTO(&file_pages, mm_v6_2, rss_stat[MM_FILEPAGES].count);
                BPF_CORE_READ_INTO(&anon_pages, mm_v6_2, rss_stat[MM_ANONPAGES].count);
                BPF_CORE_READ_INTO(&shmem_pages, mm_v6_2, rss_stat[MM_SHMEMPAGES].count);
#endif

Andreagit97 · 2024-03-21T08:13:36Z

I wonder if we could change the criteria from mm_rss_stat to kernel version?

Unfortunately no, because these changes could be backported in older kernel versions (so before 6.2.0), many distros do similar things... so the best way to detect these changes is to check the struct existence...

yzewei · 2024-03-22T09:18:43Z

I wonder if we could change the criteria from mm_rss_stat to kernel version?

Unfortunately no, because these changes could be backported in older kernel versions (so before 6.2.0), many distros do similar things... so the best way to detect these changes is to check the struct existence...

After debugging, it was found that there was no exception when running the bpf_core_type_exists () function. For higher kernels that do not have mm_rss_stat structures, the result returned by the loongarch platform is 0.
But I still can't figure out why logic is true here

yzewei · 2024-03-22T09:39:24Z

@Andreagit97
I found that the result of judging mm_rss_stat on loongarch is correct, just need to judge under the macro definition to be normal

        int result = bpf_core_type_exists(struct mm_rss_stat);
        /* In recent kernel versions (https://github.com/torvalds/linux/commit/f1a7941243c102a44e8847e3b94ff4ff3ec56f25)
         * `struct mm_rss_stat` doesn't exist anymore.
         */
#if result == 1
                BPF_CORE_READ_INTO(&file_pages, mm, rss_stat.count[MM_FILEPAGES].counter);
                BPF_CORE_READ_INTO(&anon_pages, mm, rss_stat.count[MM_ANONPAGES].counter);
                BPF_CORE_READ_INTO(&shmem_pages, mm, rss_stat.count[MM_SHMEMPAGES].counter);
#else
                struct mm_struct___v6_2 *mm_v6_2 = (void *)mm;
                BPF_CORE_READ_INTO(&file_pages, mm_v6_2, rss_stat[MM_FILEPAGES].count);
                BPF_CORE_READ_INTO(&anon_pages, mm_v6_2, rss_stat[MM_ANONPAGES].count);
                BPF_CORE_READ_INTO(&shmem_pages, mm_v6_2, rss_stat[MM_SHMEMPAGES].count);
#endif

Andreagit97 · 2024-03-22T15:16:23Z

ei @yzewei i think you have 2 possible solutions as explained here #1719 (comment);

introduce an #if defined(__TARGET_ARCH_loongarch) to add the missing structs definitions that the compiler needs.
generate an old vmlinux.h for long arch (for old I mean a kernel version <= 6.2.0, probably <= 6.0 is even better since there are other structs that are changed) and put it in the repo.

If possible I would go for solution number 2, it would be easier for everyone since it should avoid ifdefs

yzewei · 2024-03-25T02:20:54Z

ei @yzewei i think you have 2 possible solutions as explained here #1719 (comment);

introduce an #if defined(__TARGET_ARCH_loongarch) to add the missing structs definitions that the compiler needs.

generate an old vmlinux.h for long arch (for old I mean a kernel version <= 6.2.0, probably <= 6.0 is even better since there are other structs that are changed) and put it in the repo.

If possible I would go for solution number 2, it would be easier for everyone since it should avoid ifdefs

@Andreagit97 Compiling with vmlinux with a kernel smaller than 6.0 solved this problem, thank you for your guidance!!

yzewei · 2024-03-25T02:49:47Z

Andreagit97 · 2024-03-25T09:12:22Z

@Andreagit97 version 6.1.0 of Linux is compatible with Loongarch starting from version 5.19. So I selected this version to compile and obtain the low-version vmlinux file.

Ok now that you have this vmlinux.h file, you should take it and come back to your initial setup with kernel version 6.8.0. Now you should add this file in a new folder loongarch here https://github.com/falcosecurity/libs/tree/master/driver/modern_bpf/definitions and you should try to compile again. This time many of the initial issues should be solved, at least the one reported in this issue. If not, please report the compilation error and the vmlinux.h you generated.

Unfortunately, this version may not support the BTF of Loongarch for the time being, which resulted in some issues during compilation.

Please note that you need to only take the vmlinux.h file from the old kernel version, the compilation should happen on your actual setup (kernel version 6.8.0). If the old machine doesn't support BTF you cannot compile the modern probe on it

yzewei · 2024-03-25T09:26:29Z

@Andreagit97 version 6.1.0 of Linux is compatible with Loongarch starting from version 5.19. So I selected this version to compile and obtain the low-version vmlinux file.

Ok now that you have this vmlinux.h file, you should take it and come back to your initial setup with kernel version 6.8.0. Now you should add this file in a new folder loongarch here https://github.com/falcosecurity/libs/tree/master/driver/modern_bpf/definitions and you should try to compile again. This time many of the initial issues should be solved, at least the one reported in this issue. If not, please report the compilation error and the vmlinux.h you generated.

Unfortunately, this version may not support the BTF of Loongarch for the time being, which resulted in some issues during compilation.

Please note that you need to only take the vmlinux.h file from the old kernel version, the compilation should happen on your actual setup (kernel version 6.8.0). If the old machine doesn't support BTF you cannot compile the modern probe on it

Yes, I did it, I didn't encounter any other errors in the bpf part at present, but some data structures or data items do not exist due to kernel version problems. Under vmlinux.h, I added it in struct_flavors, let's see what problems will appear in the future

yzewei · 2024-03-25T09:33:17Z

@Andreagit97 Here's another question:

/home/yzw/rd/libs/userspace/libpman/src/lifecycle.c: in function ‘pman_save_attached_progs’:
/home/yzw/rd/libs/userspace/libpman/src/lifecycle.c:44:76: error：‘struct <ambigous>’not member named sched_p_fork’
   44 |       g_state.attached_progs_fds[5] = bpf_program__fd(g_state.skel->progs.sched_p_fork);
      |                                                                          ^

Andreagit97 · 2024-03-26T09:38:21Z

ei @yzewei it seems like you have defined CAPTURE_SCHED_PROC_FORK somewhere for powerpc... it's difficult to say what you have done without seeing the code... can you open a PR with the changes so we can see it?

yzewei · 2024-03-26T12:13:31Z

@Andreagit97 This problem is solved because system calls can be made under loongarch: sys_exit_execve
No need to add CAPTURE_SCHED_PROC_FORK

yzewei · 2024-03-26T12:15:25Z

But there is still a problem when making bpf, the following is the problem situation

In file included from /home/yzw/rd/libs/driver/bpf/probe.c:27:
/home/yzw/rd/libs/driver/bpf/fillers.h:2369:48: warning: passing 'volatile long *' to parameter of type 'long *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
 2369 |                 res = bpf_accumulate_argv_or_env(data, argv, &args_len);
      |                                                              ^~~~~~~~~
/home/yzw/rd/libs/driver/bpf/fillers.h:1923:19: note: passing argument to parameter 'args_len' here
 1923 |                                                       long *args_len)
      |                                                             ^
1 warning generated.
LLVM ERROR: Branch target out of insn range
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: llc -march=bpf -filetype=obj -o /home/yzw/rd/libs/driver/bpf/probe.o /home/yzw/rd/libs/driver/bpf/probe.ll
 #0 0x00007fffe8a61acc llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/lib64/libLLVM-18.so+0xa61acc)
 #1 0x00007fffe8a5f9a0 llvm::sys::RunSignalHandlers() (/lib64/libLLVM-18.so+0xa5f9a0)
 #2 0x00007fffe8a5fd88 (/lib64/libLLVM-18.so+0xa5fd88)
 #3 0x00007ffffe79c89c (linux-vdso.so.1+0x89c)
 #4 0x00007fffeffaaf58 __pthread_kill_implementation.constprop.0 (/lib64/libc.so.6+0x82f58)
 #5 0x00007fffeff651e0 raise (/lib64/libc.so.6+0x3d1e0)
 #6 0x00007fffeff502f4 abort (/lib64/libc.so.6+0x282f4)
 #7 0x00007fffe89a4d74 llvm::report_fatal_error(llvm::StringRef, bool) (/lib64/libLLVM-18.so+0x9a4d74)
 #8 0x00007fffe89a4dec llvm::install_bad_alloc_error_handler(void (*)(void*, char const*, bool), void*) (/lib64/libLLVM-18.so+0x9a4dec)
 #9 0x00007fffeb34d19c (/lib64/libLLVM-18.so+0x334d19c)
#10 0x00007fffea214eec llvm::MCAssembler::layout(llvm::MCAsmLayout&) (/lib64/libLLVM-18.so+0x2214eec)
#11 0x00007fffea215060 llvm::MCAssembler::Finish() (/lib64/libLLVM-18.so+0x2215060)
#12 0x00007fffe93ba8c4 llvm::AsmPrinter::doFinalization(llvm::Module&) (/lib64/libLLVM-18.so+0x13ba8c4)
#13 0x00007fffe8baa138 llvm::FPPassManager::doFinalization(llvm::Module&) (/lib64/libLLVM-18.so+0xbaa138)
#14 0x00007fffe8bb53e4 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/lib64/libLLVM-18.so+0xbb53e4)
#15 0x00005555587cff38 (/usr/bin/llc+0x17f38)
#16 0x00005555587c7bb4 main (/usr/bin/llc+0xfbb4)
#17 0x00007fffeff5082c __libc_start_call_main (/lib64/libc.so.6+0x2882c)
#18 0x00007fffeff50918 __libc_start_main@@GLIBC_2.36 (/lib64/libc.so.6+0x28918)
#19 0x00005555587c8140 _start (/usr/bin/llc+0x10140)
make[7]: *** [/home/yzw/rd/libs/driver/bpf/Makefile:56: /home/yzw/rd/libs/driver/bpf/probe.o]  Segmentation fault (core dumped)
make[6]: *** [/home/yzw/workspace/linux-chenhuacai/Makefile:1921：/home/yzw/rd/libs/driver/bpf] error 2
make[5]: *** [Makefile:240：__sub-make] error 2
make[4]: *** [Makefile:39：all] error 2
make[3]: *** [driver/bpf/CMakeFiles/bpf.dir/build.make:70：driver/bpf/CMakeFiles/bpf] error 2
make[2]: *** [CMakeFiles/Makefile2:859：driver/bpf/CMakeFiles/bpf.dir/all] error 2
make[1]: *** [CMakeFiles/Makefile2:866：driver/bpf/CMakeFiles/bpf.dir/rule] error 2
make: *** [Makefile:312：bpf] error 2

yzewei · 2024-03-26T12:16:20Z

ei @yzewei it seems like you have defined CAPTURE_SCHED_PROC_FORK somewhere for powerpc... it's difficult to say what you have done without seeing the code... can you open a PR with the changes so we can see it?

At present, the code needs to be sorted out and reviewed internally, so it will take some time to update the relevant code on the previous PR. Please understand, thank you very much！

yzewei · 2024-04-18T12:35:41Z

@Andreagit97 The problem has been identified, because the branch instructions compiled by llc need to encode offsets greater than 2^16 instructions, which is beyond the limit of cpuv3 and requires the use of cpuv4, and then depends on the higher version of the kernel. Other methods are being communicated so that the lower version kernel can use the bpf module of libs

Andreagit97 · 2024-04-19T10:13:25Z

oh, great catch! thank you for the update!

poiana · 2024-07-18T16:08:53Z

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

Andreagit97 · 2024-07-19T07:24:04Z

can we close this?

poiana · 2024-08-18T10:09:36Z

Stale issues rot after 30d of inactivity.

Mark the issue as fresh with /remove-lifecycle rotten.

Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle rotten

poiana · 2024-09-17T10:10:46Z

Rotten issues close after 30d of inactivity.

Reopen the issue with /reopen.

Mark the issue as fresh with /remove-lifecycle rotten.

Provide feedback via https://github.com/falcosecurity/community.
/close

poiana · 2024-09-17T10:10:49Z

@poiana: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue with /reopen.

Mark the issue as fresh with /remove-lifecycle rotten.

Provide feedback via https://github.com/falcosecurity/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

yzewei added the kind/bug Something isn't working label Mar 1, 2024

yzewei mentioned this issue Mar 2, 2024

update kernel check for modern-bpf #1723

Closed

yzewei mentioned this issue Mar 18, 2024

feat(driver): Add loongarch64 kernel module support #1654

Closed

Andreagit97 self-assigned this Mar 20, 2024

Andreagit97 added this to the TBD milestone Mar 20, 2024

poiana added the lifecycle/stale label Jul 18, 2024

poiana added lifecycle/rotten and removed lifecycle/stale labels Aug 18, 2024

poiana closed this as completed Sep 17, 2024

[BUG] Compile modern_bpf error #1719

[BUG] Compile modern_bpf error #1719

Comments

yzewei commented Mar 1, 2024 • edited Loading

Tasks

Tasks

Andreagit97 commented Mar 5, 2024

yzewei commented Mar 5, 2024

Andreagit97 commented Mar 6, 2024

yzewei commented Mar 7, 2024 • edited Loading

yzewei commented Mar 7, 2024

Andreagit97 commented Mar 7, 2024

yzewei commented Mar 19, 2024

Andreagit97 commented Mar 20, 2024 • edited Loading

yzewei commented Mar 20, 2024

yzewei commented Mar 20, 2024

yzewei commented Mar 20, 2024 • edited Loading

Andreagit97 commented Mar 20, 2024

yzewei commented Mar 20, 2024 • edited Loading

yzewei commented Mar 20, 2024

Andreagit97 commented Mar 20, 2024

yzewei commented Mar 20, 2024 • edited Loading

Andreagit97 commented Mar 20, 2024

yzewei commented Mar 21, 2024

yzewei commented Mar 21, 2024 • edited Loading

Andreagit97 commented Mar 21, 2024

yzewei commented Mar 22, 2024

yzewei commented Mar 22, 2024 • edited Loading

Andreagit97 commented Mar 22, 2024 • edited Loading

yzewei commented Mar 25, 2024 • edited Loading

yzewei commented Mar 25, 2024 • edited Loading

Andreagit97 commented Mar 25, 2024

yzewei commented Mar 25, 2024

yzewei commented Mar 25, 2024

Andreagit97 commented Mar 26, 2024

yzewei commented Mar 26, 2024

yzewei commented Mar 26, 2024

yzewei commented Mar 26, 2024 • edited Loading

yzewei commented Apr 18, 2024

Andreagit97 commented Apr 19, 2024

poiana commented Jul 18, 2024

Andreagit97 commented Jul 19, 2024

poiana commented Aug 18, 2024

poiana commented Sep 17, 2024

poiana commented Sep 17, 2024

yzewei commented Mar 1, 2024 •

edited

Loading

yzewei commented Mar 7, 2024 •

edited

Loading

Andreagit97 commented Mar 20, 2024 •

edited

Loading

yzewei commented Mar 20, 2024 •

edited

Loading

yzewei commented Mar 20, 2024 •

edited

Loading

yzewei commented Mar 20, 2024 •

edited

Loading

yzewei commented Mar 21, 2024 •

edited

Loading

yzewei commented Mar 22, 2024 •

edited

Loading

Andreagit97 commented Mar 22, 2024 •

edited

Loading

yzewei commented Mar 25, 2024 •

edited

Loading

yzewei commented Mar 25, 2024 •

edited

Loading

yzewei commented Mar 26, 2024 •

edited

Loading