SELinux in production — the workflow that actually works, and the AVC denials we keep finding

The most consistent SELinux story we hear from new customers: "we had it on briefly, something broke, we set it to permissive, that was three years ago." The host is still technically labelled. The policy is technically loaded. The kernel is decisively ignoring all of it. The compliance team has not been told.

SELinux is the single most powerful mandatory access control system shipping in mainstream Linux, and it is also the single most disabled security control we encounter. The reason is almost always the same — people try to debug SELinux problems by guessing rather than by reading the audit log. This post is the SELinux workflow we run on every managed RHEL host, the AVC denial debugging steps in order, and the small set of custom policy patterns that cover 90% of real-world cases.

The premise: enforcing or off, nothing in between

permissive mode is a debugging tool, not an operating mode. A host in permissive mode is generating audit records but enforcing nothing. The kernel sees a denied access, logs it, and lets it proceed. An attacker exploits the same access and the same thing happens — log, allow.

Our rule: every production RHEL host is SELINUX=enforcing in /etc/selinux/config. The only legitimate reason to be in permissive mode is the short window during a policy debug session. If you find a host in permissive that isn't actively being debugged, treat it as a security incident, fix the underlying policy issue, and put it back in enforcing.

# Verify
sestatus
# Expected:
# SELinux status:                 enabled
# Current mode:                   enforcing
# Loaded policy name:             targeted

Targeted policy: what's actually confined

RHEL ships the targeted policy by default. This confines a known set of system services (httpd, sshd, postgresql, named, etc.) and leaves most user processes running in the unconfined_t domain. The mls/strict policies exist but we don't use them outside specific compliance regimes — targeted is the right balance of "stops a compromised service from owning the box" without "every application breaks."

The two questions that matter:

# What domain is a process running in?
ps -eZ | grep nginx
 
# What context does a file have?
ls -Z /var/www/html/index.html

If either of those returns unconfined_t for a service that should be confined, the service was started in a way that escaped the policy — usually a custom systemd unit that didn't inherit the right context. We see this constantly with home-grown Go services that wrap nginx.

The debugging workflow, in order

When SELinux blocks something, here is the order we work through. The cardinal sin is jumping straight to audit2allow — that's step 4, not step 1.

Step 1: confirm SELinux is actually the cause

# Set permissive, retry the operation. If it still fails, SELinux isn't the issue.
setenforce 0
# ... reproduce the problem ...
setenforce 1

This is not the same as turning SELinux off forever. It is a 30-second test to confirm causation. About 30% of "SELinux is blocking us" tickets are actually unrelated permission problems (file mode 600 owned by the wrong user), and setenforce 0 proves it cleanly.

Step 2: read the audit log

ausearch -m AVC,USER_AVC -ts recent

Every SELinux denial generates an AVC (Access Vector Cache) record. The record contains the process, the target object, the requested access, and the SELinux contexts of both. A typical AVC:

type=AVC msg=audit(1715750000.123:456): avc:  denied  { read } for
pid=2345 comm="nginx" name="config.json" dev="dm-0" ino=789012
scontext=system_u:system_r:httpd_t:s0
tcontext=unconfined_u:object_r:user_home_t:s0
tclass=file permissive=0

Read this. Don't skip to a tool. The record tells you exactly what happened: nginx (httpd_t) tried to read a file labelled user_home_t, and SELinux said no. The fix is either to relabel the file (it's misplaced) or to allow the access (the policy is wrong for this site). Almost always the answer is relabel.

Step 3: check the labelling first

ls -Z /path/to/file
matchpathcon /path/to/file

matchpathcon tells you what context the policy thinks this path should have. If ls -Z shows a different context, the file is mislabelled. The fix is restorecon:

restorecon -Rv /var/www/html

Roughly 70% of all SELinux denials we see are mislabelled files. The file was copied with cp instead of mv (which preserves vs inherits contexts), or restored from backup, or written into a directory the daemon shouldn't be writing to. restorecon fixes it; the underlying habit ("use cp -Z when contexts matter") is harder to fix.

Step 4: check booleans

SELinux has dozens of boolean toggles that flip pre-baked policy decisions on and off. Whatever your "weird custom requirement" is, there is probably already a boolean for it.

getsebool -a | grep httpd
# httpd_can_network_connect_db --> off
# httpd_can_sendmail --> off
# httpd_use_nfs --> off

We see "nginx can't talk to the database" constantly, and the fix is one boolean:

setsebool -P httpd_can_network_connect_db on

The -P makes it persistent. Without it, the setting is lost on reboot, which is a great way to introduce a Heisenbug.

Step 5: only now consider a custom policy module

If labels are right, booleans don't cover it, and the access genuinely needs to be allowed, then you write a custom policy module.

# Capture the denials
ausearch -m AVC -ts recent | audit2allow -M mycustomapp
 
# Review the generated .te file BEFORE installing
cat mycustomapp.te
 
# Install if it looks reasonable
semodule -i mycustomapp.pp

The "review before installing" step is essential. audit2allow will happily generate a policy that grants system_r:httpd_t the ability to write anywhere under /, because that's what the denials asked for. That's a worse policy than no SELinux at all. Read what it wrote, narrow it down, and install only the minimal access the application actually needs.

A real custom module — the pattern we use most

The most common custom module pattern we ship is "this application writes to a non-standard log directory." For example, a customer's bespoke service running as httpd_t writes logs to /opt/acme/logs/ instead of /var/log/. The right answer is not to allow httpd_t to write everywhere — it's to label /opt/acme/logs/ correctly.

# Add the file context permanently
semanage fcontext -a -t httpd_log_t "/opt/acme/logs(/.*)?"
 
# Apply it now
restorecon -Rv /opt/acme/logs

Two commands. No custom module. The application now writes to a httpd_log_t-labelled directory, which the existing policy already allows. This is the SELinux equivalent of "use the right tool" — we add file contexts to fit existing types far more often than we write new policy.

Custom ports

The other extremely common case: a service wants to listen on a non-standard port. Postgres on 5433 instead of 5432, nginx on 8080 in addition to 80, sshd on 2222 for jump-host duty.

# Verify what's allowed
semanage port -l | grep ssh
# ssh_port_t       tcp      22
 
# Add port 2222
semanage port -a -t ssh_port_t -p tcp 2222

If you forget this, the service fails to bind with a confusing "permission denied" error and a fresh AVC in the audit log. The fix is one command, but you have to know it exists.

The booleans we toggle most often

For reference, the booleans that come up in real customer environments:

# Web servers
setsebool -P httpd_can_network_connect_db on
setsebool -P httpd_can_network_connect on
setsebool -P httpd_unified on             # only if app needs to write content
 
# Containers
setsebool -P container_manage_cgroup on
setsebool -P virt_use_nfs on
 
# SSH-based jump hosts
setsebool -P ssh_keysign on
 
# Postgres in non-standard locations
setsebool -P postgresql_can_rsync on

Default-off is correct for all of these. Enable them when you have a documented reason; document the reason in the host's configuration so the next engineer doesn't wonder why.

What we monitor

Every managed RHEL host ships AVC denials to a SIEM. The two alerting rules that matter:

• Any AVC denial in a production domain that isn't in our known-good list — fire to the on-call, this is either a misconfiguration or an actual attack
• A burst of denials from the same process in a short window — fire to security, this is a process trying to do things it isn't allowed to do, repeatedly, which often means an exploit attempt

The second one is the server-under-threat trigger. When httpd starts trying to read /etc/shadow and SELinux is denying it, the policy did its job — but you should still know.

What we don't recommend

• setenforce 0 as the first response to anything. As above. It's a debugging tool, not an operating mode.
• Disabling SELinux at boot via selinux=0. This skips even labelling, which means you can't easily turn it back on later — you'll need a full relabel, which is an outage.
• Allowing unconfined_t to do anything in a custom policy. If you find yourself writing allow unconfined_t ..., you're solving the wrong problem.
• Custom policies generated by audit2allow without review. As discussed — this is how SELinux turns into security theatre.

The summary

SELinux is not a system to fear; it's a system to operate. The workflow is: confirm causation, read the AVC, fix the label, check for a boolean, and only then write policy. The debugging steps are in order, and if you follow them you'll resolve 95% of SELinux issues without ever leaving enforcing mode.

For customers who want SELinux verified and tuned across an existing fleet — or as part of a pen-testing engagement — that's an audit we do often. We typically find a handful of permissive hosts, a dozen mislabelled directories, and a couple of legitimately misconfigured services. The cleanup is small and worth doing.

Sudhanshu K. leads cybersecurity engagements at EdgeServers (RemotIQ Pty Ltd, ABN 91 682 628 128). She has been writing SELinux policy since RHEL 5 and still believes targeted is the right default for 99% of workloads.