How to install ZFS

[sgohl]

Describe the enhancement

I am trying, without success, to get ZFS anyhow working on coreos.

I don't need it for root/boot, just for additional disks.

Is there any option I am just not aware of? I found and tested a gist which uses koji (but does not have the current kernel available), but I think then my systems break on the next zincati update.

I would also go with zfs-fuse if there's no other way, but that systemd-unit errors:

Nov 14 19:20:47 test zfs-fuse-helper[2284]: /usr/sbin/zfs-fuse-helper: line 4: /etc/rc.d/init.d/functions: No such file or directory
Nov 14 19:20:47 test zfs-fuse-helper[2286]: /usr/sbin/zfs-fuse-helper: line 22: pidofproc: command not found
Nov 14 19:20:47 test zfs-fuse-helper[2284]: Huh?

I guess that rpm-package is not compatible with coreos and I'd rather have it as kernel module, anyway.

Although I've not done it before, I would consider building a custom coreos via cosa, if that's the last option?

Thanks in advance for any help!

System details

bare metal / vmware esx

Additional information

No response

[travier]

How are you getting the ZFS source / pacakges? One option is to create a akmods package for the module so that it gets rebuilt on update. Another one is to build via a container before updates.

[sgohl]

I'm not sure if I should link the gist here, because it's from someone else. It downloads the source from https://github.com/openzfs/zfs/releases/download

But as said, that script does not work at all for me.

akmod sounds good, but I lack the complete knowledge how to put that into coreos systems

Another one is to build via a container before updates.

that sounds like zincati has a pre-update hook?

I would be happy with anything that does not involve custom coreos builds or breaking zincati updates ^^

[sgohl]

since you mentioned akmod - I found a related github issue: openzfs/zfs#12641 where is stated that akmod is not available on coreos - did that change in the meantime?

[dustymabe]

@sgohl - there's not an easy answer to this problem.

I will point out that I think this use case (out of tree kernel modules) might be a good case for OSTree native containers (also known as CoreOS Layering). We should add an example of how to build an out of tree kernel module and use that to https://github.com/coreos/coreos-layering-examples

[jmarrero]

I created a issue a few days back: coreos/layering-examples#40

I will try to get an example ready during this week.

[travier]

An example has been pushed in https://github.com/coreos/layering-examples/tree/main/build-zfs-module

[sgohl]

I just tried this and I am possibly missing a step or do it completely wrong

I used an existing coreos machine to build the image (using docker)

coreos ~/layering-examples/build-zfs-module > docker build -t myregistry.acme.corp/pub/coreos:latest -f Containerfile  .
...
docker push myregistry.acme.corp/pub/coreos:latest
...
coreos ~ > rpm-ostree rebase ostree-unverified-registry:myregistry.acme.corp/pub/coreos:latest --experimental
Pulling manifest: ostree-unverified-image:docker://docker.unitedprint.rdb/pub/coreos:latest
error: remote error: getting username and password: 1 error occurred:
	* reading JSON file "/run/containers/62011/auth.json": open /run/containers/62011/auth.json: permission denied

what's the authentication it's talking about?

And what I don't understand: I always use the quay.io/coreos/coreos-installer:release image to install a bare-metal system or to get the vmware-ova to install on vsphere esx.

Is it even possible to produce an image that can be installed with coreos-installer ?

[jmarrero]

@sgohl try loging in to your private repository using /etc/ostree/auth.json as your auth file path. i.e.

podman login --authfile /etc/ostree/auth.json myregistry.acme.corp

This is a gap in our doc that will be addressed soon: coreos/rpm-ostree#4180

As for generating bootable images from layered images, this is another open enhancement/issue we are tracking here: #1151

[cgwalters]

• reading JSON file "/run/containers/62011/auth.json": open /run/containers/62011/auth.json: permission denied

For unauthenticated images, this is this issue which is fixed in https://github.com/coreos/rpm-ostree/releases/tag/v2022.15

[sgohl]

in my case, the image is in the /pub space of the registry, so no auth is requested or required.

perfect example of a bug that'd be caught by "real" systems testing

yeah, we all have been there :-D

I wish you would consider including the zfs kernel module in the official coreos build - it wouldn't be very much disk space, i think. I know of some people who did not choose CoreOS because of missing ZFS, also ...

[dustymabe]

I wish you would consider including the zfs kernel module in the official coreos build - it wouldn't be very much disk space, i think. I know of some people who did not choose CoreOS because of missing ZFS, also ...

That's a Fedora issue, not a Fedora CoreOS one. Believe me, there are a lot of people that would love for ZFS to be included in the upstream kernel (i.e. licensing issues resolved).

[sgohl]

licensing issues

oh right, there was something with CDDL

But as I said initially, I would absolutely try to create my own coreos iso/ova/whatever, if I only knew how to do it. These tools like cosa are a book of seven seals to me, unfortunately.

Docs seem to require that you already have base knowlege, but lack examples.

An example how to do it on a coreos machine would be superior. I'm not sure if I like this container native ostree approach, but I still don't understand half of it yet since I was stuck at the first attempt, but I know, this is out of the scope of this issue 😆

[dustymabe]

Since there has been an example pushed for this I don't see much else to do here. Closing.

[ItalyPaleAle]

I am struggling to get this to work, following the example. I can see the RPMs installed in the layer with rpm -qa. But when I try to use the commands I get an error such as:

$ sudo zfs list
The ZFS modules cannot be auto-loaded.
Try running 'modprobe zfs' as root to manually load them.

And modprobe:

$ sudo modprobe zfs
modprobe: FATAL: Module zfs not found in directory /lib/modules/6.5.11-300.fc39.x86_64

For reference:

$ uname -a
Linux atlas 6.5.11-300.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Nov  8 22:37:57 UTC 2023 x86_64 GNU/Linux

$ rpm -qa | grep zfs
zfs-debugsource-2.2.2-1.fc39.x86_64
zfs-debuginfo-2.2.2-1.fc39.x86_64
libzfs5-2.2.2-1.fc39.x86_64
zfs-kmod-debugsource-2.2.2-1.fc39.x86_64
kmod-zfs-6.5.11-300.fc39.x86_64-2.2.2-1.fc39.x86_64
zfs-2.2.2-1.fc39.x86_64
kmod-zfs-6.5.11-300.fc39.x86_64-debuginfo-2.2.2-1.fc39.x86_64
libzfs5-debuginfo-2.2.2-1.fc39.x86_64

-- EDIT

I figured out the issue: had to run depmod. I opened a PR to fix this: coreos/layering-examples#71