1. Schedule & House Keeping

We are hoping to keep this to a tight 50 minutes or so. There is a TON of content here. So if we don’t get to the end, this page isn’t going anywhere, and I answer tweets all the time.

  • Confirm you can get to your lab instance (5 minutes)

  • Key Concepts Presentation (10-15 minutes)

  • Lab Exercises (30-35 minutes)

Tip If you have a question or if something doesn’t work, be loud and we’ll get it figured out.

2. Lab Instance Preparation

You should have a VM somewhere where you have full root privileges. If you don’t, you can follow along up here, but it won’t be as much fun.

2.1. Root Access

ssh in as root, or ssh in and then elevate to a root prompt using sudo.

ssh in as root
jduncan@dhcp-192-168-1-140 ~$ ssh root@selinux-lab.example.com
ssh in as another user and elevate to root
jduncan@dhcp-192-168-1-140 ~$ ssh cloud-user@selinux-lab.example.com
cloud-user@selinux-lab.example.com's password:
[cloud-user@selinux-lab ~]$ su -
Last login: Fri Feb  5 11:57:16 EST 2016 from on pts/0
[root@selinux-lab ~]#
Important You may need to be on the VCU VPN to access your guest instance

3. Lab Configuration

3.1. Demo System

I am using a kvm virtual machine on my laptop for this lab. It is running RHEL 7.2, patched as of 8am 09-Feb-2016 (today).

current kernel and selinux versions
[root@selinux-lab ~]# rpm -qa | grep '^kernel\|selinux'
Important You’re likely using CentOS, and that’s fine for today. But do know that they are NOT the same thing. It’s a fun conversation if we have time. If you’re using Ubuntu, just know that AppArmor IS THE ABSOLUTE WORST.

4. SELinux Modes

SELinux can be either in the enabled or disabled state. When disabled, only DAC rules are used. When enabled, SELinux can run in one of the following modes:


SELinux policy is not enforced. SELinux does not deny access, but denials are logged for actions that would have been denied if running in enforcing mode.

— SELinux User Guide for RHEL 7

Enforcing: SELinux policy is enforced. SELinux denies access based on SELinux policy rules.

— SELinux User Guide for RHEL 7

This is not really a mode, but actually tells the kernel not to use the SELinux modules at all. THIS IS SUPER-HIGHLY NOT RECOMMENDED.

5. Ensure SELinux is Enforcing

[root@selinux-lab ~]# getenforce
if SELinux is in Permissive mode, set it to Enforcing
[root@selinux-lab ~]# getenforce
[root@selinux-lab ~]# setenforce 1
[root@selinux-lab ~]# getenforce
if SELinux is disabled, go to /etc/sysconfing/selinux and set the SELINUX parameter to enforcing
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=enforcing 1
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
1 The SELINUX parameter
if you have to change this value, you will need to reboot
[root@selinux-lab ~]# systemctl reboot
Tip If SELinux is disabled by default, you should also find your senior admins and threaten them with great harm for making you insecure for no good reason.

6. This Workshop As Code

This entire workshop is available in GitHub. It is written in asciidoc, and published with asciidoctor through a Jenkins job.

Note Pull Requests and Stars are both very welcome.

7. Key Concepts Presentation

The key concepts around SELinux will be presented via a presentation. Once that is over we will move to this agenda and have a live mini-workshop.

The presentation is available at slides.com.

Important This link will follow along live with my presentation. Your laptop is not possessed.

8. Apache Lab

8.1. Start and Enable httpd

enable httpd via systemctl
[root@selinux-lab ~]# systemctl enable httpd
start httpd
[root@selinux-lab ~]# systemctl start httpd
confirm httpd is running
[root@selinux-lab ~]# systemctl status httpd
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2016-02-08 13:39:22 EST; 1min 11s ago
     Docs: man:httpd(8)
 Main PID: 11305 (httpd)
   Status: "Total requests: 0; Current requests/sec: 0; Current traffic:   0 B/sec"
   CGroup: /system.slice/httpd.service
           ├─11305 /usr/sbin/httpd -DFOREGROUND
           ├─11306 /usr/sbin/httpd -DFOREGROUND
           ├─11307 /usr/sbin/httpd -DFOREGROUND
           ├─11308 /usr/sbin/httpd -DFOREGROUND
           ├─11309 /usr/sbin/httpd -DFOREGROUND
           ├─11310 /usr/sbin/httpd -DFOREGROUND
           └─11311 /usr/sbin/httpd -DFOREGROUND

8.2. Configure httpd

In RHEL/CentOS the default location for web content is /var/www/html. So let’s create a simple file there so we can tell if it works.

create /var/www/html/index.txt
[root@selinux-lab ~]# echo 'this is my default directory' > /var/www/html/index.txt
confirm that httpd is serving this content
[root@selinux-lab ~]# curl
this is my default directory
Tip This would also work in a browser pointed at your remote server. This is the interwebs, after all. I’m just an old CLI guy, and it’s easier for you to copy/paste to go through the lab.

8.3. Serve httpd Content From Another Directory

8.3.1. Problem Statement

You have now received word that you need to have https serve its web content from another location. It is uber-important to the application developers that all content come from /var/webapp.

make a new directory to serve content from
[root@selinux-lab ~]# mkdir /var/webapp
create some content
[root@selinux-lab ~]# echo 'this is my webapp directory' > /var/webapp/index.txt
tell httpd to serve content from our new directory
[root@selinux-lab ~]# sed -i 's/www\/html/webapp/' /etc/httpd/conf/httpd.conf
restart httpd
[root@selinux-lab ~]# systemctl restart httpd
test out our new configuration
[root@selinux-lab ~]# curl
<title>403 Forbidden</title>
<p>You don't have permission to access /index.txt
on this server.</p>
[root@selinux-lab ~]# curl
<title>403 Forbidden</title>
<p>You don't have permission to access /index.txt
on this server.</p>

Uh Oh…​

8.4. Troubleshooting httpd access issues

check /var/log/httpd/error.log
[root@selinux-lab ~]# tail /var/log/httpd/error_log
[Mon Feb 08 14:58:07.019620 2016] [core:error] [pid 11544] (13)Permission denied: [client] AH00035: access to /index.txt denied (filesystem path '/var/webapp/index.txt') because search permissions are missing on a component of the path
make sure modes and ownerships are correct on the parent directories
[root@selinux-lab ~]# ll /var/www/
total 0
drwxr-xr-x. 2 root root  6 Sep 17 09:07 cgi-bin
drwxr-xr-x. 2 root root 22 Feb  8 13:42 html
[root@selinux-lab ~]# ll /var/
total 12
drwxr-xr-x.  2 root root   22 Feb  8 14:52 webapp
make sure modes and ownership are correct on the files
[root@selinux-lab ~]# ll /var/www/html/index.txt
-rw-r--r--. 1 root root 29 Feb  8 13:42 /var/www/html/index.txt
[root@selinux-lab ~]# ll /var/webapp/index.txt
-rw-r--r--. 1 root root 28 Feb  8 14:52 /var/webapp/index.txt
check the SELinux context of the files
[root@selinux-lab ~]# ll -Z /var/www/html/index.txt
-rw-r--r--. root root unconfined_u:object_r:httpd_sys_content_t:s0 /var/www/html/index.txt
[root@selinux-lab ~]# ll -Z /var/webapp/index.txt
-rw-r--r--. root root unconfined_u:object_r:var_t:s0   /var/webapp/index.txt

So SELinux is not allowing httpd to read files that have a var_t type. This is a good thing, since it is the default value given to anything in the /var directory that doesn’t have an SELinux Context defined.

Tip If a file isn’t accounted for by one of the policies in targeted mode, it will be given an unconfined_t type

8.5. Using chcon

chcon is used to change the SELinux Context of an object.

change the file’s context to httpd_sys_content_t
[root@selinux-lab ~]# chcon -R --type=httpd_sys_content_t /var/webapp/
ensure your change took effect
[root@selinux-lab ~]# ll -Z /var/webapp/index.txt
drwxr-xr-x. root root unconfined_u:object_r:httpd_sys_content_t:s0 /var/webapp/index.txt
see if the new SELinux context allows us access to the file through httpd
[root@selinux-lab ~]# curl
this is my webapp directory
Important SELinux contexts for files and directories are stored in the filesystem’s extended attributes. So they are persistent over a reboot.

8.6. Creating a new SELinux Context

8.6.1. Problem Statement

We just got word from our webapp team. They now need to server all content from /var/webapp2. Before we go hunt them down and beat them with Nerf swords let’s get the work done for what they need.

create the new directory
[root@selinux-lab ~]# mkdir /var/webapp2
create /var/webapp2/index.txt
[root@selinux-lab ~]# echo 'this is my second webapp page' > /var/webapp2/index.txt
alter our apache configs …​ again
[root@selinux-lab ~]# sed -i 's/webapp/webapp2/' /etc/httpd/conf/httpd.conf
restart apache …​ again
[root@selinux-lab ~]# systemctl restart httpd
test to see if it is accessible
[root@selinux-lab ~]# curl
<title>403 Forbidden</title>
<p>You don't have permission to access /index.txt
on this server.</p>

8.6.2. Using semanage

semanage is a command-line tool used to manage SELinux contexts effectively on a system. Our primary concern today is how to add contexts that will be picked up and used by the targeted policy on our system.

The semanage fcontext command is used to change the SELinux context of files. When using targeted policy, changes are written to files located in the /etc/selinux/targeted/contexts/files/ directory.

— RHEL 7 SELinux Guide
Tip The man pages for semanage and all other SELinux-related userspace tools are incredibly useful and robust.
the situation before making any changes
[root@selinux-lab ~]# ll -Z /var/
drwxr-xr-x. root root unconfined_u:object_r:httpd_sys_content_t:s0 webapp
drwxr-xr-x. root root unconfined_u:object_r:var_t:s0   webapp2
create a new context for /var/webapp2
[root@selinux-lab ~]# semanage fcontext -a -t httpd_sys_content_t "/var/webapp2(/.*)?"
check /etc/selinux/targeted/contexts/files/file_contexts.local
[root@selinux-lab ~]# grep webapp2 /etc/selinux/targeted/contexts/files/file_contexts.local
/var/webapp2(/.*)?    system_u:object_r:httpd_sys_content_t:s0
note that /var/webapp2 still has the wrong context
[root@selinux-lab ~]# ll -Z /var/
drwxr-xr-x. root root unconfined_u:object_r:var_t:s0   webapp2

8.6.3. Using restorecon to reset defaults

Now that we have a context defined, we can restore the defaults, and it should become what we need it to be.

run restorecon on /var/webapp2
[root@selinux-lab ~]# restorecon -R -v /var/webapp2
restorecon reset /var/webapp2 context unconfined_u:object_r:var_t:s0->unconfined_u:object_r:httpd_sys_content_t:s0
restorecon reset /var/webapp2/index.txt context unconfined_u:object_r:var_t:s0->unconfined_u:object_r:httpd_sys_content_t:s0
confirm that we can now access our content
[root@selinux-lab ~]# curl
this is my second webapp page

8.7. Apache Lab Recap

We were able to
  • Use chcon to make immediate changes to the SELinux context for files on a filesystem

  • Use semanage fcontext to create a new SELinux context for our system

  • Use restorecon to apply our newly created default values for our system

  • Troubleshoot the issues through apache and into the kernel and filesystem

9. Additional SELinux Tools

9.1. Booleans

Booleans allow parts of SELinux policy to be changed at runtime, without any knowledge of SELinux policy writing. This allows changes, such as allowing services access to NFS volumes, without reloading or recompiling SELinux policy.

— RHEL 7 SELinux Guide

9.1.1. List available Booleans

[root@selinux-lab ~]# semanage boolean -l
SELinux boolean                State  Default Description

ftp_home_dir                   (off  ,  off)  Allow ftp to home dir
smartmon_3ware                 (off  ,  off)  Allow smartmon to 3ware
mpd_enable_homedirs            (off  ,  off)  Allow mpd to enable homedirs
xdm_sysadm_login               (off  ,  off)  Allow xdm to sysadm login
xen_use_nfs                    (off  ,  off)  Allow xen to use nfs
mozilla_read_content           (off  ,  off)  Allow mozilla to read content
ssh_chroot_rw_homedirs         (off  ,  off)  Allow ssh to chroot rw homedirs
mount_anyfile                  (on   ,   on)  Allow mount to anyfile
cron_userdomain_transition     (on   ,   on)  Allow cron to userdomain transition
icecast_use_any_tcp_ports      (off  ,  off)  Allow icecast to use any tcp ports
openvpn_can_network_connect    (on   ,   on)  Allow openvpn to can network connect
zoneminder_anon_write          (off  ,  off)  Allow zoneminder to anon write
minidlna_read_generic_user_content (off  ,  off)  Allow minidlna to read generic user content
spamassassin_can_network       (off  ,  off)  Allow spamassassin to can network
how many booleans on my system
[root@selinux-lab ~]# semanage boolean -l | wc -l

9.1.2. Some Personal Favorites

By favorites I mean things that have killed me in the past.

virt_sandbox_use_nfs (off , off) Allow virt to sandbox use nfs

This one tripped me up for 2 hours just last week. It allows Linux Containers to mount NFS volumes directly for shared storage. Much pain was had until I found this buried down in some docs somewhere.

use_nfs_home_dirs (off , off) Allow use to nfs home dirs

This is needed if you automount your homedirs from an NFS share.

httpd_enable_homedirs (off , off) Allow httpd to enable homedirs

This lets your serve http content from your home directory

Tip people.redhat.com/~jduncan/$this_workshop is apache being served from my home directory on a server somewhere. It therefore relies on this boolean being enabled.

9.2. system-config-selinux

If you absolutely, positively, have to have a GUI…​

funny pictures cat computer popup
Figure 1. random kitty image

That’s all I have to say about that.

10. SELinux Logging

One of the hardest things to understand is the cryptic nature of how SELinux logs what it does. You have to remember that this is all happening down in the kernel, where every CPU cycle matters, so it errs to speed instead of readability.

one of our denials from earlier
type=AVC msg=audit(1454963764.269:497): avc:  denied  { getattr } for  pid=11670 comm="httpd" path="/var/webapp2/index.txt" dev="dm-0" ino=101094487 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:var_t:s0 tclass=file
the denial log entry broken down
type=AVC 1
msg=audit(1454963764.269:497): 2
avc:  denied  { getattr } for  pid=11670 3
comm="httpd" 4
path="/var/webapp2/index.txt" dev="dm-0" ino=101094487 5
scontext=system_u:system_r:httpd_t:s0 6
tcontext=unconfined_u:object_r:var_t:s0 7
tclass=file 8
1 This is an AVC alert
2 epoch time stamp
[root@selinux-lab ~]# date -d @1454963764.269
Mon Feb  8 15:36:04 EST 2016
1 this was a denial for PID 11670 trying to run a 'getattr'
2 process that caused the denial
3 path of the target
4 context of the process (domain)
5 context of the target
6 class of the target (a file in this case)
In English

At Mon Feb 8 15:36:04 EST 2016 httpd running as PID 11670 tried to access inode 101094487 on device dm-0 (/var/webapp2/index.txt) and was denied by SELinux because the file had a var_t SELinux type.

11. Further Resources