cross-built kernel/dkms improvements #46332

classabbyamp · 2023-09-29T21:58:58Z

dkms: allow limiting jobs, add missing dep
- some less-powerful platforms may be overwhelmed by the default -j$(nproc). Allow overriding this by setting DKMS_JOBS in /etc/default/dkms
- bc is needed for make prepare0 on kernel 6.1+, which is run on cross.
- see also: rpi-kernel: update to 6.1.54 (and rpi-firmware, rpi-userland, rpi-eeprom) #46152 (comment)
linux6.1: include files necessary for dkms on cross
linux6.5: include files necessary for dkms on cross

The same fix could be applied to linux6.3 and linux6.4, but with the impending zfs 2.1.13 bump (#46304), I don't see much point.

fixes #44807

Testing the changes

I tested the changes in this PR: YES (tested in an aarch64-musl VM on 6.3 with several dkms modules)

@moabeat-berlin, @r-ricci, @Calandracas606: please test

[ci skip]

Calandracas606 · 2023-09-30T16:46:16Z

Was able to install ddcci-dkms for a natively compiled 6.3.13_2 kernel on my aarch64-musl system

zfs dkms failing, but there's no hard errors in the /var/lib/dkms/zfs/2.1.12/build/build/build.log, although there is a warning

modpost: "blk_register_queue" [/var/lib/dkms/zfs/2.1.12/build/build/bio_bdev_disk/bio_bdev_disk.ko] undefined!

and many "Skipping BTF generation for /var/lib/dkms/zfs/2.1.12/build/build/foobar/foobar.ko due to unavailability of vmlinux"

The issue with the missing headers appears to have been resolved though

classabbyamp · 2023-09-30T16:54:06Z

can you try rm /lib/modules/<version>/build/scripts/basic/fixdep then xbps-reconfigure -f linux6.3 to ensure that triggers one of the previous fail conditions

Calandracas606 · 2023-09-30T17:05:40Z

Getting error linux6.3-6.3.13_2: configuring ...
Executing post-install kernel hook: 10-dkms ...
HOSTCC scripts/basic/fixdep
UPD include/generated/compile.h
scripts/Makefile.build:41: arch/arm64/tools/Makefile: No such file or directory
make[1]: *** No rule to make target 'arch/arm64/tools/Makefile'. Stop.
make: *** [arch/arm64/Makefile:179: archprepare] Error 2

moabeat-berlin · 2023-09-30T18:21:41Z

I think I got something. I cross-built "linux6.1" and "linux6.5" packages for "aarch64" and "aarch64-musl" from this PR. If you would like to pull them, you can get them from https://barrens.de/repo/, e.g. xbps-install -S -R https://barrens.de/repo linux6.5-headers. linux6.3 is still running.

My findings executing make prepare0 for both of these headers packages on "aarch64-musl" so far:

arch/arm64/tools/* is missing multiple files, maybe just keep the directory entirely.
arch/arm64/kernel/asm-offsets.c is required
include/kvm/* is missing.

After adding them, the command succeeds. @Calandracas606, you seem to run into the same issue with 6.3.

Shouldn't actually everything in include/* be part of the "*-headers" packages? Excluding include/generated of course.

Will test dkms after that got fixed, but probably we are good after adding those files.

moabeat-berlin · 2023-09-30T22:29:58Z

Packages for "linux6.3" are also available now. The same files are also missing there. When they have been added, dkms seems to be working fine.

installing dkms package "ddcci-dkms"

# xbps-install ddcci-dkms

Name       Action    Version           New version            Download size
ddcci-dkms install   -                 0.4.3_1                22KB 

Size to download:               22KB
Size required on disk:          82KB
Space available on disk:        24GB

Do you want to continue? [Y/n] 

[*] Downloading packages
ddcci-dkms-0.4.3_1.aarch64-musl.xbps.sig2: 512B [avg rate: 4425KB/s]
ddcci-dkms-0.4.3_1.aarch64-musl.xbps: 22KB [avg rate: 115MB/s]
ddcci-dkms-0.4.3_1: verifying RSA signature...

[*] Collecting package files
ddcci-dkms-0.4.3_1: collecting files...

[*] Unpacking packages
ddcci-dkms-0.4.3_1: unpacking ...

[*] Configuring unpacked packages
ddcci-dkms-0.4.3_1: configuring ...
Added DKMS module 'ddcci-0.4.3'.
Prepare to build modules for kernel-6.1.54_1... done.
Building DKMS module 'ddcci-0.4.3' for kernel-6.1.54_1... done.
Installing DKMS module 'ddcci-0.4.3' for kernel-6.1.54_1... done.
Skipping kernel-6.1.55_2. kernel-headers package not installed...
Prepare to build modules for kernel-6.3.13_2... done.
Building DKMS module 'ddcci-0.4.3' for kernel-6.3.13_2... done.
Installing DKMS module 'ddcci-0.4.3' for kernel-6.3.13_2... done.
ddcci-dkms-0.4.3_1: installed successfully.

1 downloaded, 1 installed, 0 updated, 1 configured, 0 removed.

classabbyamp · 2023-10-01T01:06:06Z

include/kvm/* is missing.

I don't think that's needed, from my testing. what needed it?

Calandracas606 · 2023-10-01T03:08:59Z

ZFS builds after changing /usr/src/zfs*/META from CDDL to GPL

some less-powerful platforms may be overwhelmed by the default `-j$(nproc)`. Allow overriding this by setting `DKMS_JOBS` in `/etc/default/dkms` `bc` is needed for `make prepare0` on kernel 6.1+, which is run on cross.

adds the ability to limit the number of makejobs to the dkms trigger

moabeat-berlin · 2023-10-01T10:02:50Z

include/kvm/* is missing.

I don't think that's needed, from my testing. what needed it?

The difference is, I added asm-offsets.c but the new version of package includes asm-offsets.s. The .s file was not part of the original linux kernel tarball, I was using as a reference and is a generated file. Regeneration obviously requires the kvm-headers.

moabeat-berlin · 2023-10-01T20:15:53Z

I have rebuilt the packages in this PR with the latest changes (aarch64 and aarch64-musl), source is still https://barrens.de/repo/. I had to remove them also from /var/cache/xbps/ to make sure it's not the old one installed from there.

Testing dkms on "aarch64-musl" by installing "ddcci-dkms" was successful and it also respected the /etc/default/dkms jobs settings on the Raspberry Pi Zero 2W (using new the "xbps-triggers" package). However, installing "zfs" was still not possible (device unresponsive after some time). It did not respect the setting. It seems to use it's own mechanism of building the module and was running with -j4 again. But I was not able to figure out yet how this exactly happens.

dkms ddcci-dkms install log

# xbps-install ddcci-dkms                                                                

Name       Action    Version           New version            Download size
ddcci-dkms install   -                 0.4.3_1                - 

Size required on disk:          82KB
Space available on disk:        27GB

Do you want to continue? [Y/n] 

[*] Verifying package integrity
ddcci-dkms-0.4.3_1: verifying RSA signature...

[*] Collecting package files
ddcci-dkms-0.4.3_1: collecting files...

[*] Unpacking packages
ddcci-dkms-0.4.3_1: unpacking ...

[*] Configuring unpacked packages
ddcci-dkms-0.4.3_1: configuring ...
Added DKMS module 'ddcci-0.4.3'.
Prepare to build modules for kernel-6.1.54_1... done.
Building DKMS module 'ddcci-0.4.3' for kernel-6.1.54_1... done.
Installing DKMS module 'ddcci-0.4.3' for kernel-6.1.54_1... done.
Prepare to build modules for kernel-6.1.55_2... done.
Building DKMS module 'ddcci-0.4.3' for kernel-6.1.55_2... done.
Installing DKMS module 'ddcci-0.4.3' for kernel-6.1.55_2... done.
Prepare to build modules for kernel-6.3.13_2... done.
Building DKMS module 'ddcci-0.4.3' for kernel-6.3.13_2... done.
Installing DKMS module 'ddcci-0.4.3' for kernel-6.3.13_2... done.
Prepare to build modules for kernel-6.5.5_2... done.
Building DKMS module 'ddcci-0.4.3' for kernel-6.5.5_2... done.
Installing DKMS module 'ddcci-0.4.3' for kernel-6.5.5_2... done.
ddcci-dkms-0.4.3_1: installed successfully.

0 downloaded, 1 installed, 0 updated, 1 configured, 0 removed.

make processes during zfs install

#ps -Af | grep make
root      2326  2325 13 20:13 pts/1    00:00:00 make modules -k -j4 CONFIG_MODULES=y CFLAGS_MODULE=-DCONFIG_MODULES -C /lib/modules/6.1.54_1/build M=/var/lib/dkms/zfs/2.1.12/build/build
moabeat   2543 14729  0 20:13 pts/2    00:00:00 grep make

So, in general probably a success, beside "zfs" on the Zero 2W.

This was referenced Sep 29, 2023

rpi-kernel: update to 6.1.54 (and rpi-firmware, rpi-userland, rpi-eeprom) #46152

Merged

DKMS Modules Fail to Build on aarch64 - Missing powerpc Headers #44807

Closed

classabbyamp force-pushed the dkms-cross branch from 7e1ae20 to babf74c Compare September 30, 2023 01:14

classabbyamp force-pushed the dkms-cross branch from babf74c to 551befc Compare October 1, 2023 01:59

classabbyamp added 5 commits October 1, 2023 00:40

dkms: allow limiting jobs, add missing dep

418c850

some less-powerful platforms may be overwhelmed by the default `-j$(nproc)`. Allow overriding this by setting `DKMS_JOBS` in `/etc/default/dkms` `bc` is needed for `make prepare0` on kernel 6.1+, which is run on cross.

xbps-triggers: update to 0.126.

aa6348a

adds the ability to limit the number of makejobs to the dkms trigger

linux6.1: include files necessary for dkms on cross

d34fe07

linux6.3: include files necessary for dkms on cross

98f48d6

linux6.5: include files necessary for dkms on cross

d51a4c7

classabbyamp force-pushed the dkms-cross branch from 551befc to d51a4c7 Compare October 1, 2023 04:40

classabbyamp merged commit 9a7b55d into void-linux:master Oct 1, 2023
2 checks passed

classabbyamp deleted the dkms-cross branch October 1, 2023 20:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cross-built kernel/dkms improvements #46332

cross-built kernel/dkms improvements #46332

classabbyamp commented Sep 29, 2023 •

edited

Loading

Calandracas606 commented Sep 30, 2023 •

edited

Loading

classabbyamp commented Sep 30, 2023 •

edited

Loading

Calandracas606 commented Sep 30, 2023

moabeat-berlin commented Sep 30, 2023

moabeat-berlin commented Sep 30, 2023

classabbyamp commented Oct 1, 2023 •

edited

Loading

Calandracas606 commented Oct 1, 2023 •

edited

Loading

moabeat-berlin commented Oct 1, 2023 •

edited

Loading

moabeat-berlin commented Oct 1, 2023

cross-built kernel/dkms improvements #46332

cross-built kernel/dkms improvements #46332

Conversation

classabbyamp commented Sep 29, 2023 • edited Loading

Testing the changes

Calandracas606 commented Sep 30, 2023 • edited Loading

classabbyamp commented Sep 30, 2023 • edited Loading

Calandracas606 commented Sep 30, 2023

moabeat-berlin commented Sep 30, 2023

moabeat-berlin commented Sep 30, 2023

classabbyamp commented Oct 1, 2023 • edited Loading

Calandracas606 commented Oct 1, 2023 • edited Loading

moabeat-berlin commented Oct 1, 2023 • edited Loading

moabeat-berlin commented Oct 1, 2023

classabbyamp commented Sep 29, 2023 •

edited

Loading

Calandracas606 commented Sep 30, 2023 •

edited

Loading

classabbyamp commented Sep 30, 2023 •

edited

Loading

classabbyamp commented Oct 1, 2023 •

edited

Loading

Calandracas606 commented Oct 1, 2023 •

edited

Loading

moabeat-berlin commented Oct 1, 2023 •

edited

Loading