Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSSD binaries missing capabilities in Bazzite 41 vs. Kinoite 41 #1818

Open
rayrayrayraydog opened this issue Oct 30, 2024 · 15 comments
Open
Labels
bug Something isn't working

Comments

@rayrayrayraydog
Copy link

Describe the bug

Upon upgrading Bazzite to F41, I found that my layered install of SSSD was broken. I also found that the same SSSD setup would work when added to a new install of Kinoite 41.

In a test VM of Bazzite from the 41 ISO, I was able to recreate this issue. I think I've narrowed it down to a few binaries used by SSSD not having capabilities assigned as they do with the same package on Kinoite.

bazzite system:

root@bazzite:/usr/libexec/sssd# getcap *
selinux_child cap_chown,cap_dac_override,cap_setgid,cap_setuid=ep

kinoite system:

root@kinotest:/usr/libexec/sssd# getcap *
krb5_child cap_chown,cap_dac_override,cap_setgid,cap_setuid=ep
ldap_child cap_chown,cap_dac_override,cap_setgid,cap_setuid=ep
selinux_child cap_chown,cap_dac_override,cap_setgid,cap_setuid=ep
sssd_pam cap_dac_read_search=p

What did you expect to happen?

I would expect the binaries krb5_child, ldap_child, and sssd_pam under /usr/libexec/sssd to have the proper capabilities so that SSSD services can function. I cannot directly modify them as they are under /usr. This was not an issue in bazzite on F39, wherein SSSD ran under the root account.

I realize that the bazzite team doesn't touch SSSD and doesn't ship it, but given that it works out of the box on Kinoite with the same version of the package I can't rule out something in bazzite's setup.

Output of rpm-ostree status

root@bazzite:/usr/libexec/sssd# rpm-ostree status
State: idle
Deployments:
● ostree-image-signed:docker://ghcr.io/ublue-os/bazzite:stable
Digest: sha256:dd21242732e272339c1e5d9cee6441f95e819223e12d53cf1430a3db517d6bd6
Version: 41.20241029.1 (2024-10-29T15:39:27Z)
LayeredPackages: adcli htop krb5-workstation libguestfs-tools libvirt libvirt-daemon-config-network libvirt-daemon-kvm
oddjob oddjob-mkhomedir plasma-workspace-x11 pugixml qemu-kvm sssd terminator virt-install virt-manager
virt-top virt-viewer

ostree-image-signed:docker://ghcr.io/ublue-os/bazzite:stable
Digest: sha256:dd21242732e272339c1e5d9cee6441f95e819223e12d53cf1430a3db517d6bd6
Version: 41.20241029.1 (2024-10-29T15:39:27Z)
LayeredPackages: adcli htop krb5-workstation libguestfs-tools libvirt libvirt-daemon-config-network libvirt-daemon-kvm
oddjob

Hardware

This is a KVM virtual machine built with the latest bazzite-stable.iso for F41.

root@bazzite:/usr/libexec/sssd# cat /sys/devices/virtual/dmi/id/product_name Standard PC (Q35 + ICH9, 2009)

Extra information or context

No response

@dosubot dosubot bot added the bug Something isn't working label Oct 30, 2024
@antheas
Copy link
Contributor

antheas commented Oct 30, 2024

Is the sssd binary new? @KyleGospo can add a seuid in the Dockerfile for now as i dont feel like bumping rechunk

Oh we never checked for that directory

@rayrayrayraydog
Copy link
Author

Is the sssd binary new? @KyleGospo can add a seuid in the Dockerfile for now as i dont feel like bumping rechunk

Oh we never checked for that directory

It was a non-issue until F41 where the service account sssd was added to run the services as non-root. It's mentioned here: https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/sysadmin/

Thank you for helping support an edge case like this.

@antheas
Copy link
Contributor

antheas commented Oct 30, 2024

@rayrayrayraydog
Copy link
Author

Noticed this morning that the SSSD package is installed by default in bazzite and Fedora in general. The file selinux_child that has capabilities comes from another package I had layered to get it working with MS AD.

@karypid
Copy link

karypid commented Nov 2, 2024

So then, if I understand correctly upgrading from f40 to f41 has some script that changes file ownership permissions somewhere? Or is this part of the new RPM post-install scripts?

This is interesting. If the latter, I would expect everything in /usr to have the proper permissions, but how would this be addressed for /etc files in atomic distros? What does Fedora Silverblue do? Perhaps the problem is present there as well?

@antheas
Copy link
Contributor

antheas commented Nov 2, 2024

We are using a derived image from kinoite that gets rechunked in the end

Onenof the quirks of doing this is that we lose the xattrs of the original image for now. If we didn't rechunk the image we would lose the derived xattrs instead

In any case, what changed in f41 is that sssd is now part of kinoite, which caused it to lose its xattrs. So we had to add them back in

Rechunk 1.0.1 is merged in testing so next bazzite build will have them fixed

@karypid
Copy link

karypid commented Nov 3, 2024

I updated Bluefin gts today and can no longer login via sssd:

AutomaticUpdates: stage; rpm-ostreed-automatic.timer: last run 41min ago
Deployments:
  ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx-nvidia:gts
                   Digest: sha256:c7d12e8d5e6bf444ffa3c0c740aaaa5bbc204360d0315b6389d96538fa8f4bb8
                  Version: 40.20241102.0 (2024-11-03T05:44:06Z)
                     Diff: 25 upgraded, 5 removed, 6 added
          LayeredPackages: touchegg

● ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx-nvidia:gts
                   Digest: sha256:dfc42faacfffa0f990206db843e9a9cd84f58a99374b7f81123ca4aeaebead4c
                  Version: 40.20241029.0 (2024-10-29T18:30:00Z)
          LayeredPackages: touchegg
                   Pinned: yes

I have rolled back to the 40.20241029.0 build which still works fine. I am not sure if something has been merged (that should not have been) in 40-based images, or maybe they are also affected and need the same fix?

Furthermore, after facing the same issue on my second system, I instead upgraded to Bluefin latest (41-based) and that seems to work fine:

AutomaticUpdates: stage; rpm-ostreed-automatic.timer: no runs since boot
Deployments:
● ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx:latest
                   Digest: sha256:27d65e684ef5e4159e480ec4c531b137579d31707773cba9d13bf75dbbf47495
                  Version: 41.20241103.0 (2024-11-03T04:44:01Z)
          LayeredPackages: pwgen

  ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx:gts
                   Digest: sha256:64fc3f87ad0249f9d6b3284c60168948e0c5a4e54cb54943b91d133abe5dfc5a
                  Version: 40.20241102.0 (2024-11-03T05:44:36Z)
          LayeredPackages: pwgen

  ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx:stable
                   Digest: sha256:75fe45926ca23fe63414bc42b22f40e650decc1eae8d6d4b621e7d8e6b3721d0
                  Version: 40.20241027.0 (2024-10-27T05:46:52Z)
          LayeredPackages: pwgen
                   Pinned: yes

I can also confirm that ownership has changed:

❯ sudo ls -al /var/lib/sss/
total 0
drwxrwxr-x. 1 sssd sssd  100 May  2  2024 .
drwxr-xr-x. 1 root root 1042 Nov  3 17:20 ..
drwxrwx---. 1 sssd sssd  394 Nov  3 17:20 db
drwxrwx--x. 1 sssd sssd    0 Apr 30  2024 deskprofile
drwxrwx---. 1 sssd sssd    0 Apr 30  2024 gpo_cache
drwx------. 1 root root    0 Apr 30  2024 keytabs
drwxrwxr-x. 1 sssd sssd   48 Nov  3 17:20 mc
drwxrwxr-x. 1 sssd sssd   32 Nov  3 17:20 pipes
drwxrwxr-x. 1 sssd sssd  108 Nov  3 17:25 pubconf
drwxrwx---. 1 sssd sssd   22 Apr 30  2024 secrets

localuser in 🌐 myhost in ~ 
❯ ps -ef | grep sssd
sssd        1550       1  0 17:20 ?        00:00:00 /usr/sbin/sssd -i --logger=files
sssd        1561    1550  0 17:20 ?        00:00:00 /usr/libexec/sssd/sssd_be --domain ad.home.lan --logger=files
sssd        1563    1550  0 17:20 ?        00:00:00 /usr/libexec/sssd/sssd_nss --logger=files
sssd        1564    1550  0 17:20 ?        00:00:00 /usr/libexec/sssd/sssd_pam --logger=files
sssd        1565    1550  0 17:20 ?        00:00:00 /usr/libexec/sssd/sssd_pac --logger=files
sssd        2966       1  0 17:20 ?        00:00:00 /usr/libexec/sssd/sssd_kcm --logger=files
localus+   31124   28469  0 17:34 pts/1    00:00:00 grep --color=auto sssd

Not sure if any fix has been merged, but just reporting the data points here in case they are useful.

@antheas
Copy link
Contributor

antheas commented Nov 3, 2024

@castrojo update to rechunk 1.0.1 to fix this

@castrojo
Copy link
Member

castrojo commented Nov 3, 2024

We bumped to 1.0.1 yesterday but that was after the stable builds went out, I'll push out new ones shortly.

@castrojo
Copy link
Member

castrojo commented Nov 3, 2024

OK @karypid - images are updated, let me know!

@rayrayrayraydog
Copy link
Author

@castrojo Do you mean a beta image? I'm not seeing anything yet.

@karypid
Copy link

karypid commented Nov 4, 2024

So, I think the problem is that the capabilities are only needed for 41-based images where the software runs as user sssd.

At least this is the only difference I can find between the 3 images below, of which only the pinned works:

❯ rpm-ostree status
State: idle
AutomaticUpdates: stage; rpm-ostreed-automatic.timer: no runs since boot
Deployments:
● ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx-nvidia:stable-daily
                   Digest: sha256:e1554f6ad79bd38c739e1175a0a45c2b6bcf5b07b37f671bf0cc345ac65755b0
                  Version: 40.20241103.0 (2024-11-04T05:46:52Z)
          LayeredPackages: touchegg

  ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx-nvidia:gts
                   Digest: sha256:ddcdb165ae06a724ee4093485b40fd4bf548a5cb22b1abf6e65aa7b09ba68a4a
                  Version: 40.20241102.0 (2024-11-03T17:58:54Z)
          LayeredPackages: touchegg

  ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx-nvidia:gts
                   Digest: sha256:dfc42faacfffa0f990206db843e9a9cd84f58a99374b7f81123ca4aeaebead4c
                  Version: 40.20241029.0 (2024-10-29T18:30:00Z)
          LayeredPackages: touchegg
                   Pinned: yes

In all 3 everything is owned and runs as root so that is common:

❯ ls -al /var/lib/sss/
total 0
drwxr-xr-x. 1 root root  100 Apr 14  2024 .
drwxr-xr-x. 1 root root 1032 May 16 19:20 ..
drwx------. 1 root root 2066 Nov  4 19:20 db
drwxr-x--x. 1 root root    0 Apr 14  2024 deskprofile
drwxr-xr-x. 1 root root    0 Apr 14  2024 gpo_cache
drwx------. 1 root root    0 Apr 14  2024 keytabs
drwxrwxr-x. 1 root root   48 Nov  4 19:20 mc
drwxr-xr-x. 1 root root   32 Nov  4 19:20 pipes
drwxr-xr-x. 1 root root   70 Nov  4 19:26 pubconf
drwx------. 1 root root   22 Apr 14  2024 secrets

~/sssd 
❯ ps -ef | grep sssd
root        1853       1  0 19:20 ?        00:00:00 /usr/sbin/sssd -i --logger=files
root        1996    1853  0 19:20 ?        00:00:00 /usr/libexec/sssd/sssd_be --domain ad.home.lan --uid 0 --gid 0 --logger=files
root        2090    1853  0 19:20 ?        00:00:00 /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --logger=files
root        2091    1853  0 19:20 ?        00:00:00 /usr/libexec/sssd/sssd_pam --uid 0 --gid 0 --logger=files
root        2092    1853  0 19:20 ?        00:00:00 /usr/libexec/sssd/sssd_pac --uid 0 --gid 0 --logger=files
root        4066       1  0 19:24 ?        00:00:00 /usr/libexec/sssd/sssd_kcm --uid 0 --gid 0 --logger=files
localus+    5327    5107  0 19:27 pts/1    00:00:00 grep --color=auto sssd

But the 40.20241102.0 (gts) and 40.20241103.0 (stable-daily) have the new capabilities which I think are interfering (these are probably only needed for 41-based images):

❯ sudo getcap /usr/libexec/sssd/*
/usr/libexec/sssd/krb5_child cap_chown,cap_dac_override,cap_setgid,cap_setuid=ep
/usr/libexec/sssd/ldap_child cap_chown,cap_dac_override,cap_setgid,cap_setuid=ep
/usr/libexec/sssd/selinux_child cap_chown,cap_dac_override,cap_setgid,cap_setuid=ep
/usr/libexec/sssd/sssd_pam cap_dac_read_search=p

The older pinned 40.20241029.0 (gts) which is the only one that works, has no capabilities:

~/sssd 
❯ sudo getcap /usr/libexec/sssd/*
[sudo] password for localuser: 

~/sssd took 3s 

As I reported yesterday, latest works fine, and has the capabilities, but is also 41-based and runs/owns the files as sssd....

@rayrayrayraydog
Copy link
Author

rayrayrayraydog commented Nov 4, 2024

As I reported yesterday, latest works fine, and has the capabilities, but is also 41-based and runs/owns the files as sssd....

That's odd. I'm on latest 41 but I don't see the capabilities there. Do you mean in Bluefin?

@karypid
Copy link

karypid commented Nov 4, 2024

As I reported yesterday, latest works fine, and has the capabilities, but is also 41-based and runs/owns the files as sssd....

That's odd. I'm on latest 41 but I don't see the capabilities there. Do you mean in Bluefin?

Ah yes: as shown in my rpm-ostree status output, I am testing on Bluefin. Latest Bluefin works fine for me (tested yesterday) and runs everything as sssd (but file ownership is also set to user sssd).

I am not clean on how much bazzite/bluefin share in the underlying infrastructure. Happy to file a separate bug report if this is not a ublue-os core issue.

@karypid
Copy link

karypid commented Nov 4, 2024

@rayrayrayraydog Hello from Bluefix-41. Just double-confirmed that latest is working fine:

When running on Bluefin 41 (latest) everything is owned by sssd except keytabs (not sure if that is an issue, but it definitely doesn't prevent me from logging in):

❯ ls -al /var/lib/sss/
total 0
drwxrwxr-x. 1 sssd sssd  100 Apr 14  2024 .
drwxr-xr-x. 1 root root 1042 Nov  4 22:53 ..
drwxrwx---. 1 sssd sssd 2094 Nov  4 22:54 db
drwxrwx--x. 1 sssd sssd    0 Apr 14  2024 deskprofile
drwxrwx---. 1 sssd sssd    0 Apr 14  2024 gpo_cache
drwx------. 1 root root    0 Apr 14  2024 keytabs
drwxrwxr-x. 1 sssd sssd   48 Nov  4 22:53 mc
drwxrwxr-x. 1 sssd sssd   32 Nov  4 22:53 pipes
drwxrwxr-x. 1 sssd sssd  108 Nov  4 22:54 pubconf
drwxrwx---. 1 sssd sssd   22 Apr 14  2024 secrets

Everything runs as sssd:

~ 
❯ ps -ef | grep sssd
sssd        2080       1  0 22:53 ?        00:00:00 /usr/sbin/sssd -i --logger=files
sssd        2150    2080  0 22:53 ?        00:00:00 /usr/libexec/sssd/sssd_be --domain ad.home.lan --logger=files
sssd        2152    2080  0 22:53 ?        00:00:00 /usr/libexec/sssd/sssd_nss --logger=files
sssd        2153    2080  0 22:53 ?        00:00:00 /usr/libexec/sssd/sssd_pam --logger=files
sssd        2154    2080  0 22:53 ?        00:00:00 /usr/libexec/sssd/sssd_pac --logger=files
sssd        3574       1  0 22:54 ?        00:00:00 /usr/libexec/sssd/sssd_kcm --logger=files
localus+   21424   20925  0 22:55 pts/1    00:00:00 grep --color=auto sssd

And the capabilities are present:

~ took 3s 
❯ sudo getcap /usr/libexec/sssd/*
/usr/libexec/sssd/krb5_child cap_chown,cap_dac_override,cap_setgid,cap_setuid=ep
/usr/libexec/sssd/ldap_child cap_chown,cap_dac_override,cap_setgid,cap_setuid=ep
/usr/libexec/sssd/selinux_child cap_chown,cap_dac_override,cap_setgid,cap_setuid=ep
/usr/libexec/sssd/sssd_pam cap_dac_read_search=p

Here is rpm-ostree status with 41.20241104.0 (latest) on bluefin-dx-nvidia channel:

~ 
❯ rpm-ostree status
State: idle
AutomaticUpdates: stage; rpm-ostreed-automatic.timer: no runs since boot
Deployments:
● ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx-nvidia:latest
                   Digest: sha256:a11ce0ebeadd60956be56cff181d656b6f43edb15fb98bc54b709c85ff332a71
                  Version: 41.20241104.0 (2024-11-04T21:02:12Z)
          LayeredPackages: touchegg

  ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx-nvidia:gts
                   Digest: sha256:dfc42faacfffa0f990206db843e9a9cd84f58a99374b7f81123ca4aeaebead4c
                  Version: 40.20241029.0 (2024-10-29T18:30:00Z)
          LayeredPackages: touchegg
                   Pinned: yes

So it seems to me that Bluefin has the opposite problem:

  • GTS/stable (40-based channels) are now broken. They still run everything as root, kept ownership to root, but added capabilities (which should not have been added for when running as root I guess?)
  • Latest (41-based channel) works fine. It switched to running as user sssd, changed ownership to the same and added the capabilities (which seem to be ok for when running as sssd)...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants