Directory Permissions with Local Users

[TProhofsky]

When configuring security to use local users, I see the two users I defined get created as local users to the pod created by the operator for the SmbShare CRD. What seems to be missing, or a mistake in my configuration, is updating file system permissions for the PVC mounted in under /mnt. After applying the yaml below I can connect to the share using either user but can not write to it because neither of my users have write permissions. I can exec in and chown or chmod to grant write access but this seems like I'm missing something. What is the intended method? It seems like the group of local users should be created by the operator and set for write access when the share is creates for the CRD.

apiVersion: v1
kind: Secret
metadata:
name: labsambausersWhen configuring security to use local users, I see the two users I defined get created as local users to the pod created by the operator for the SmbShare CRD. What seems to be missing, or a mistake in my configuration, is updating file system permissions for the PVC mounted in under /mnt. After applying the yaml below I can connect to the share using either user but can not write to it because neither of my users have write permissions. I can exec in and chown or chmod to grant write access but this seems like I'm missing something. What is the intended method? It seems like the group of local users should be created by the operator and set for write access when the share is creates for the CRD.

apiVersion: v1
kind: Secret
metadata:
name: labsambausers
type: Opaque
stringData:
demousers: |
{
"samba-container-config": "v0",
"users": {
"all_entries": [
{
"name": "demo",
"password": "demo"
},
{
"name": "webber",
"password": "webber"
}
]
}
}

apiVersion: samba-operator.samba.org/v1alpha1
kind: SmbSecurityConfig
metadata:
name: myusers
spec:
mode: user
users:
secret: labsambausers
key: demousers

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: sambainlab
spec:
accessModes:

• ReadWriteOnce
  volumeMode: Filesystem
  storageClassName: simpleclass
  resources:
  requests:
  storage: 7Gi

---

apiVersion: samba-operator.samba.org/v1alpha1
kind: SmbShare
metadata:
name: labshare
spec:
securityConfig: myusers
storage:
pvc:
name: "sambainlab"
readOnly: false

type: Opaque
stringData:
demousers: |
{
"samba-container-config": "v0",
"users": {
"all_entries": [
{
"name": "demo",
"password": "demo"
},
{
"name": "webber",
"password": "webber"
}
]
}
}

apiVersion: samba-operator.samba.org/v1alpha1
kind: SmbSecurityConfig
metadata:
name: myusers
spec:
mode: user
users:
secret: labsambausers
key: demousers

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: sambainlab
spec:
accessModes:

• ReadWriteOnce
  volumeMode: Filesystem
  storageClassName: simpleclass
  resources:
  requests:
  storage: 7Gi

---

apiVersion: samba-operator.samba.org/v1alpha1
kind: SmbShare
metadata:
name: labshare
spec:
securityConfig: myusers
storage:
pvc:
name: "sambainlab"
readOnly: false

[phlogistonjohn]

This is a good point, thanks for bringing it up. Currently, the operator and the samba instance pods don't really do anything to ensure that the permissions on the pvc are usable, they just take whatever is available. I think but I have not double checked that the default permissions vary depending on what provisioned the PVC. I usually test with ceph based volumes using rook,etc and I think that the permissions I get by default are permissive. But IIRC when I used a cloud provider's volume provisioner directly once I had to tweak the permissions first.

We haven't invested much time into permissions yet because we've been pushing for enhancements to samba to permit the use of NT-style ACLs without the use of the security xattr namespace because we don't want to require the containers to run with the CAP_SYS_ADMIN capability. But I'd be open to improving sambacc and the operator to improve permissions support in the near-term. I'd be very welcome to contributions in the area too! :-)

[TProhofsky]

Understanding that this issue wasn't cockpit error I have dug in a bit more on what should be the desired behavior. At the heart of the issue is that permissions need to be set on the CSI mounted point as the pod is getting started. The question is who (or what role) should set what GID? Starting with 'who' I don't think AD or DC is the best path through sambacc since the pod isn't running. We could give temporary ownership to a GID that then gets update when the container starts and has AD access. Then Active Directory is the who with some schema for associating mount GIDs to shares. This feels like adding unwanted complexity for limited security gains. Without AD we are left with the node's LDAP local user management or Kubernetes security context. Banking on LDAP seems like it could limit some deployments. My logic thus far then is to have the 'who' be the K8s orchestrator for the mount permission and then AD or local user definitions can manage all file level permissions below. Since we don't want the fsgroup to be set recursively, wiping out user permissions, then I think we should plan on using the set fsgroup enhancement: https://github.com/gnufied/enhancements/blob/master/keps/sig-storage/2317-fsgroup-on-mount/README.md

Now to what GID? Are GID collisions of shared pools of storage a multitenancy issue or attack surface? Not sure yet. If we let the human pick any a GID like 1234 then I believe the PVC definition needs to have fsgroup=1234 and the pod started by the operator needs the runAsGroup=1234. Then I thing the change to SIK is adding a runasgroup parameter to the smbShare CRD.

Does this sound close to right?

[phlogistonjohn]

Understanding that this issue wasn't cockpit error I have dug in a bit more on what should be the desired behavior. At the heart of the issue is that permissions need to be set on the CSI mounted point as the pod is getting started. The question is who (or what role) should set what GID?

A reasonable question. I think the answer will vary a bit depending on the use case for the share and if the pvc is "private" to the share or if there's some expectation that the pvc will be accessed by something other than the samba pods. Note that so far, all the code and testing has been oriented towards the case where the pvc is private to the share.

Starting with 'who' I don't think AD or DC is the best path through sambacc since the pod isn't running. We could give temporary ownership to a GID that then gets update when the container starts and has AD access.

It's a possibility. Right now we assume: (a) the processes in the container are running as root (it may or may not be it's own user namespace) (b) running without CAP_SYS_ADMIN (c) root can write to pvc. If either (a) or (c) are untrue then we'd never have the ability to even set user-specific permissions/acls. Because samba uses the ability to "become" the users that are logging in I thought that (a) was a requirement. But I could be missing some interactions. I do know that OpenShift limits this by default but you can opt in to running as uid 0 and we plan to enable that when using OpenShift.

Then Active Directory is the who with some schema for associating mount GIDs to shares. This feels like adding unwanted complexity for limited security gains.

Unfortunately, I'm not clear on what you mean here. I agree that adding something specific to AD for k8s/samba would be undesirable, but I'm not sure what you might be adding.

Without AD we are left with the node's LDAP local user management or Kubernetes security context. Banking on LDAP seems like it could limit some deployments.

Last I looked into k8s own LDAP integration it basically just handles checking k8s user's passwords / dynamic user mapping. Plus the k8s users don't directly map to uids/gids at the posix level. However, I'm not 100% sure that is the kind of LDAP integration you're referring to.

My logic thus far then is to have the 'who' be the K8s orchestrator for the mount permission and then AD or local user definitions can manage all file level permissions below.

If I interpret you correctly here, I agree. We need the processes within the container to be able to attain read-write access to the volume and then we can set permissions+ACLs specific to the users needs. The latter half of that is very similar to a traditional non-containerized Samba server.

Since we don't want the fsgroup to be set recursively, wiping out user permissions, then I think we should plan on using the set fsgroup enhancement: https://github.com/gnufied/enhancements/blob/master/keps/sig-storage/2317-fsgroup-on-mount/README.md

Yes, that's been on my radar. Thanks for reminding me.

On top of that I want to provide "good" defaults the when shares are first added/initialized but allow the (admin) users to get/set acls even on the share root. For a previous codebase I had set an xattr on the root of the share to indicate it has already been initialized if the xattr is not present the init container(s) can set the default ACL and then set the marker xattr. If the xattr is present the ACLs are left untouched.

Now to what GID? Are GID collisions of shared pools of storage a multitenancy issue or attack surface? Not sure yet. If we let the human pick any a GID like 1234 then I believe the PVC definition needs to have fsgroup=1234 and the pod started by the operator needs the runAsGroup=1234. Then I thing the change to SIK is adding a runasgroup parameter to the smbShare CRD.

Does this sound close to right?

I'll have to look into this issue with regards to the pvc access by gid. Since the back-end I am using defaults to "open" I was not acutely aware of this issue. Like I mentioned above, I thought running as uid 0 would resolve some of these kinds of issues that typical k8s applications should be constrained by. However, I'm willing to admit I may have overlooked something.
I also want to alert @synarete to this discussion - he's been looking into some of this stuff wrt openshift.

Thanks very much for this detailed discussion. We have a lot to follow up on! Please don't hesitate to continue the conversation! :-)

[TProhofsky]

Let me first disclose I have only been a light user of samba long ago so don't assume I developer understanding. With that said I still see a missing step in what you described. Here's an example to setup the question. I'm have a 6 node v1.23 cluster started using Kubespray on bare metal. The nodes are SAS connected to a shared JBOD of SSDs using a CSI driver capable of provisioning LVs as PVCs out of a StorageClass using a shared LVM2 Volume Group.

I added a SmbShare CRD using the yaml posted earlier so the PVC is private to the pod. Exec'ing into the pod I can see the PVC and its directory under /mnt.

[root@node1 try]# kubectl exec -it labshare-6fdc6b7dbc-6bbd6 -- bash
[root@labshare-6fdc6b7dbc-6bbd6 /]# df
Filesystem                        1K-blocks     Used Available Use% Mounted on
overlay                           492590080 46621372 445968708  10% /
tmpfs                                 65536        0     65536   0% /dev
tmpfs                              16154480        0  16154480   0% /sys/fs/cgroup
tmpfs                              31944416        4  31944412   1% /run
/dev/sda5                         492590080 46621372 445968708  10% /share
/dev/sbvg_prow/csilv320636oz18trg   6281216    76904   6204312   2% /mnt/283900eb-f95a-4a30-b72d-c1bd8275928a
tmpfs                              31944416        4  31944412   1% /etc/container-users
shm                                   65536        0     65536   0% /dev/shm
tmpfs                              31944416       12  31944404   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs                              16154480        0  16154480   0% /proc/acpi
tmpfs                              16154480        0  16154480   0% /proc/scsi
tmpfs                              16154480        0  16154480   0% /sys/firmware

Here are the permissions on the mounted logical volume:

[root@labshare-6fdc6b7dbc-6bbd6 /]# stat /mnt/283900eb-f95a-4a30-b72d-c1bd8275928a/
  File: /mnt/283900eb-f95a-4a30-b72d-c1bd8275928a/
  Size: 6               Blocks: 0          IO Block: 4096   directory
Device: fd11h/64785d    Inode: 128         Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 1970-01-01 00:00:00.000000000 +0000
Modify: 2022-04-14 15:06:11.210048000 +0000
Change: 2022-04-14 15:06:11.210048000 +0000
 Birth: 2022-04-14 15:06:11.210048000 +0000

The operator created the SAMBA user demo and webber:

[root@labshare-6fdc6b7dbc-6bbd6 /]# cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:65534:65534:Kernel Overflow User:/:/sbin/nologin
tss:x:59:59:Account used for TPM access:/dev/null:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/usr/sbin/nologin
systemd-oom:x:999:999:systemd Userspace OOM Killer:/:/usr/sbin/nologin
systemd-resolve:x:193:193:systemd Resolver:/:/usr/sbin/nologin
systemd-timesync:x:998:998:systemd Time Synchronization:/:/usr/sbin/nologin
systemd-coredump:x:997:997:systemd Core Dumper:/:/usr/sbin/nologin
demo:x:1000:1000::/invalid:/bin/false
webber:x:1001:1001::/invalid:/bin/false

Using smbclient to connect I can see what would be expected (but not desired) in this state that user webber cannot write to this file system.

[root@node1 ~]# smbclient -U SAMBA/webber //10.233.70.11/labshare
Enter SAMBA\webber's password:
Try "help" to get a list of possible commands.
smb: \> mkdir homebase
NT_STATUS_ACCESS_DENIED making remote directory \homebase

If the logged in samba users is "becoming" webber in this example, what do you expect gives webber permission to create a directory in the mounted filesystem base presented as the share?

[phlogistonjohn]

Thanks a bunch for the detailed steps and command outputs. It does help me a lot. And, apologies for the slow reply.

As I think you surmised earlier, the issue lies with the default permissions the CSI provisioner/mounter is creating on the mount. Your stat command clearly shows permissions 755 with owner root/root. This is an entirely reasonable default for the CSI components to provide.

What we need to do is set the permissions to something appropriate for multi-user smbd access. I think the simplest thing to do is have the sambacc tool set useful permissions on the mount point (share root) before smbd starts. In the future this could also be changed to set useful NT ACLs. As a stop-gap a teammate of mine added a custom init container to the pod to change the permissions. However, if you're not comfortable touching the code you'll probably have to wait a little bit.

[spuiuk]

I've hit the same problem using minikube and rook. The permission of the mount point was set to 0755. This change seems to be recent. Earlier versions of my cluster had the mount point set to 0777. Setting the mode to 0777 allowed me to write to the share as expected.

[spuiuk]

It turns out that the problem seen on my setup is related to an issue with setting up the rook-cephfs operator. I wasn't getting shared volumes and was instead relying on the standard storage class which was setup with incorrect permissions.