Extension to provide SMB and CIFS DFS Referral Support (keyutils and cifs-utils)

[jpuskar]

Description

On Talos Linux, we have found that pods cannot mount SMB/CIFS paths when served via DFS-N.
This is a feature request to add an extension which would provide support.

Repro steps:

1. Build a samba vm (or use existing windows DFS-N):
   1. Build an ubuntu vm called samba-server
   2. sudo apt update && apt install vim samba iputils-ping dnsutils net-tools -y
   3. replace /etc/samba/smb.conf content with this

      [global]
         workgroup = WORKGROUP
         load printers = no
         server string = Samba DFS Server
         dns proxy = no
         map to guest = Bad User
         smb ports = 445
         host msdfs = yes
         usershare allow guests = yes
         security = user
         guest account = nobody
         log level = 3
      
      [dfsroot]
         path = /srv/dfsroot
         browseable = yes
         read only = no
         guest ok = yes
         msdfs root = yes
      
      [dfsreplica]
         path = /srv/dfsreplica
         browseable = yes
         read only = no
         guest ok = yes

   4. sudo mkdir -p /srv/dfsroot /srv/dfsreplica
   5. ln -s "msdfs:samba-server\\dfsreplica" /srv/dfsroot/testshare
   6. systemctl restart smbd
2. Use this spec to create a pod (change samba_ip to the ip of your samba server):

   ---
   apiVersion: v1
   kind: Pod
   metadata:
     name: cifs-debug
     namespace: kube-system
   spec:
     hostPID: true
     restartPolicy: Never
     containers:
       - name: shell
         image: ubuntu:22.04
         securityContext:
           privileged: true
           runAsUser: 0
           capabilities:
             add:
               - SYS_ADMIN
               - SYS_PTRACE
         command:
           - bash
           - -c
           - |
             apt update && apt install -y \
               vim procps wget iputils-ping dnsutils \
               smbclient cifs-utils net-tools \
               && echo "_samba_ip_ samba-server" >> /etc/hosts \
               && mkdir -p /mnt/dfs \
               && echo "Startup complete." \
               && sleep infinity
         volumeMounts:
           - name: sys
             mountPath: /sys
           - name: proc
             mountPath: /proc
     volumes:
       - name: proc
         hostPath:
           path: /proc
           type: Directory
       - name: sys
         hostPath:
           path: /sys
           type: Directory

3. Shell into the pod and run the following mount command:

   mount \
     -t cifs \
     //samba-server/dfsroot/testshare \
     /mnt/dfs \
     -o guest

Expected results:

# no output (mount is completed) 

Actual Results:

mount error(2): No such file or directory
mount error(126): Required key not available

Analysis:

When mounting a DFS share, the kernel authenticates to the DFS server. But when the kernel follows the DFS referral, the kernel is presented with an additional auth challenge from the 'actual' target SMB server.
The kernel then:

1. kernel calls /sbin/request-key
2. /sbin/request-key then:
   1. opens and parses /etc/request-key.conf
   2. executes /usr/bin/cifs.upcall (per /etc/request-key.conf)
   3. executes keyctl to add the key
3. kernel continues the mount process

The error seems to stem from the fact that the binaries named request-key, cifs.upcall, and keyctl, are missing.
These cannot be provided inside the container namespace, because this is a call from kernel-space back down to
user-space, and the kernel does not seem to be able to do this in a namespace-aware manner.

Proposal:

To fix this, it seems like we will need:

1. An extension which adds the binaries request-key and keyctl from keyutils, and cifs.upcall from cifs-utils.
   1. Note: the request-key source will need patched to look for /etc/request-key.conf in a talos-approved location.
2. A Kernel patch so that the kernel calls /sbin/request-key in a talos-approved path.

Logs

Pod logs for a failed smb mount.

MountVolume.MountDevice failed for volume "test-volume-smb-test-volume-files-1" : rpc error: code = Internal desc = volume(test-volume-smb-test-volume-files-1) mount "//contoso.com/data/dir/loc/env/PATH" on "/var/lib/kubelet/plugins/kubernetes.io/csi/smb.csi.k8s.io/efacee789a3b68628fce73b783603113e51076334f0e12a3cc318b9e5e54af07/globalmount" failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t cifs -o dir_mode=0775,file_mode=0775,domain=CONTOSO,<masked> //contoso.com/data/dir/loc/env/PATH /var/lib/kubelet/plugins/kubernetes.io/csi/smb.csi.k8s.io/efacee789a3b68628fce73b783603113e51076334f0e12a3cc318b9e5e54af07/globalmount Output: mount error(2): No such file or directory Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel log messages (dmesg)
MountVolume.MountDevice failed for volume "test-volume-smb-test-volume-files-1" : rpc error: code = Internal desc = volume(test-volume-smb-test-volume-files-1) mount "//contoso.com/data/dir/loc/env/PATH" on "/var/lib/kubelet/plugins/kubernetes.io/csi/smb.csi.k8s.io/efacee789a3b68628fce73b783603113e51076334f0e12a3cc318b9e5e54af07/globalmount" failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t cifs -o dir_mode=0775,file_mode=0775,domain=CONTOSO,<masked> //contoso.com/data/dir/loc/env/PATH /var/lib/kubelet/plugins/kubernetes.io/csi/smb.csi.k8s.io/efacee789a3b68628fce73b783603113e51076334f0e12a3cc318b9e5e54af07/globalmount Output: mount error(126): Required key not available Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel log messages (dmesg)

Demsg logs from the node hosting the pod.

10.66.211.41: kern:  notice: [2025-07-11T18:45:57.845599374Z]: CIFS: enabling forceuid mount option implicitly because uid= option is specified
10.66.211.41: kern:  notice: [2025-07-11T18:45:57.845620374Z]: CIFS: enabling forcegid mount option implicitly because gid= option is specified
10.66.211.41: kern:    info: [2025-07-11T18:45:57.845625374Z]: CIFS: Attempting to mount //contoso.com/data/dir/loc/env/PATH
10.66.211.41: kern:     err: [2025-07-11T18:45:58.807196374Z]: CIFS: VFS: cifs_mount failed w/return code = -126

Environment

• talos version

Client:
Tag:         v1.9.5
SHA:         d07f6daa
Built:
Go version:  go1.23.7
OS/Arch:     linux/amd64
Server:
NODE:        10.66.211.34
Tag:         v1.9.5
SHA:         d07f6daa
Built:
Go version:  go1.23.7
OS/Arch:     linux/amd64
Enabled:     RBAC

• kubernetes version

Client Version: v1.32.2
Kustomize Version: v5.5.0
Server Version: v1.32.3