This section covers NFS volume security, including matching permissions and
SELinux considerations. The reader is expected to understand the basics of POSIX
permissions, process UIDs, supplemental groups, and SELinux.
Developers request NFS storage by referencing, in the volumes section of
their pod definition, either a PVC by name or the NFS volume plug-in directly.
The /etc/exports file on the NFS server contains the accessible NFS
directories. The target NFS directory has POSIX owner and group IDs. The
OpenShift Origin NFS plug-in mounts the container’s NFS directory with the same
POSIX ownership and permissions found on the exported NFS directory. However,
the container is not run with its effective UID equal to the owner of the NFS
mount, which is the desired behavior.
As an example, if the target NFS directory appears on the NFS server as:
# ls -lZ /opt/nfs -d
drwxrws---. nfsnobody 5555 unconfined_u:object_r:usr_t:s0 /opt/nfs
# id nfsnobody
uid=65534(nfsnobody) gid=65534(nfsnobody) groups=65534(nfsnobody)
Then the container must match SELinux labels, and either run with a UID of
65534 (nfsnobody owner) or with 5555 in its supplemental groups in order
to access the directory.
|
|
The owner ID of 65534 is used as an example. Even though NFS’s root_squash
maps root (0) to nfsnobody (65534), NFS exports can have arbitrary owner
IDs. Owner 65534 is not required for NFS exports.
|
Group IDs
The recommended way to handle NFS access (assuming it is not an option to change
permissions on the NFS export) is to use supplemental groups. Supplemental
groups in OpenShift Origin are used for shared storage, of which NFS is an
example. In contrast, block storage, such as Ceph RBD or iSCSI, use the
fsGroup SCC strategy and the fsGroup value in the pod’s
securityContext.
|
|
It is generally preferable to use supplemental group IDs to gain access to
persistent storage versus using user IDs. Supplemental
groups are covered further in the full
Volume Security topic.
|
Because the group ID on the example target NFS directory shown
above is 5555, the pod can define that group ID using supplementalGroups
under the pod-level securityContext definition. For example:
spec:
containers:
- name:
...
securityContext: (1)
supplementalGroups: [5555] (2)
| 1 |
securityContext must be defined at the pod level, not under a specific container. |
| 2 |
An array of GIDs defined for the pod. In this case, there is one element in the array;
additional GIDs would be comma-separated. |
Assuming there are no custom SCCs that might satisfy the pod’s requirements, the
pod will likely match the restricted SCC. This SCC has the
supplementalGroups strategy set to RunAsAny, meaning that any supplied
group ID will be accepted without range checking.
As a result, the above pod will pass admissions and will be launched. However,
if group ID range checking is desired, a custom SCC, as described in
pod security and custom
SCCs, is the preferred solution. A custom SCC can be created such that minimum
and maximum group IDs are defined, group ID range checking is enforced, and a
group ID of 5555 is allowed.
User IDs
User IDs can be defined in the container image or in the pod definition. The
full Volume Security topic covers
controlling storage access based on user IDs, and should be read prior to
setting up NFS persistent storage.
|
|
It is generally preferable to use supplemental
group IDs to gain access to persistent storage versus using user IDs.
|
In the example target NFS directory shown above, the container
needs its UID set to 65534 (ignoring group IDs for the moment), so the following
can be added to the pod definition:
spec:
containers: (1)
- name:
...
securityContext:
runAsUser: 65534 (2)
| 1 |
Pods contain a securtityContext specific to each container (shown here) and
a pod-level securityContext which applies to all containers defined in the pod. |
| 2 |
65534 is the nfsnobody user. |
Assuming the default project and the restricted SCC, the pod’s requested
user ID of 65534 will, unfortunately, not be allowed, and therefore the pod will
fail. The pod fails because of the following:
-
It requests 65534 as its user ID.
-
All SCCs available to the pod are examined to see which SCC will allow a user ID
of 65534 (actually, all policies of the SCCs are checked but the focus here is
on user ID).
-
Because all available SCCs use MustRunAsRange for their runAsUser
strategy, UID range checking is required.
-
65534 is not included in the SCC or project’s user ID range.
It is generally considered a good practice not to modify the predefined SCCs.
The preferred way to fix this situation is to create a custom SCC, as described
in the full Volume Security topic.
A custom SCC can be created such that minimum and maximum user IDs are defined,
UID range checking is still enforced, and the UID of 65534 will be allowed.
SELinux
|
|
See the full Volume Security
topic for information on controlling storage access in conjunction with using
SELinux.
|
By default, SELinux does not allow writing from a pod to a remote NFS server.
The NFS volume mounts correctly, but is read-only.
To enable writing to NFS volumes with SELinux enforcing on each node, run:
# setsebool -P virt_use_nfs 1
# setsebool -P virt_sandbox_use_nfs 1
The -P option above makes the bool persistent between reboots.
The virt_use_nfs boolean is defined by the docker-selinux package. If an
error is seen indicating that this bool is not defined, ensure this package has
been installed.
Export Settings
In order to enable arbitrary container users to read and write the volume, each
exported volume on the NFS server should conform to the following conditions:
-
Each export must be:
/<example_fs> *(rw,root_squash,no_wdelay)
The no_wdelay option prevents the server from delaying writes, which greatly
improves read-after-write consistency.
-
The firewall must be configured to allow traffic to the mount point. For NFSv4,
the default port is 2049 (nfs). For NFSv3, there are three ports to configure:
2049 (nfs), 20048 (mountd), and 111 (portmapper).
NFSv4
# iptables -I INPUT 1 -p tcp --dport 2049 -j ACCEPT
NFSv3
# iptables -I INPUT 1 -p tcp --dport 2049 -j ACCEPT
# iptables -I INPUT 1 -p tcp --dport 20048 -j ACCEPT
# iptables -I INPUT 1 -p tcp --dport 111 -j ACCEPT
-
The NFS export and directory must be set up so that it is accessible by the
target pods. Either set the export to be owned by the container’s primary UID,
or supply the pod group access using supplementalGroups, as shown in
Group IDs above. See the full
Volume Security topic for additional pod
security information as well.