Configuring SELinux: Services Fail After Enabling Restrictions
Enabling mandatory access control on a node that has run without it for years rarely goes smoothly.
The web server stops reading the static directory, the database does not come up after a restart, and scheduled jobs finish silently — a service failure after enabling looks like a random set of crashes, although they have the same cause.
Policy describes not users but processes: which service is allowed to open a file, bind a port, access a socket.
Labels on objects are applied during creation and copying, so the directory where a release is deployed can easily get the label of a temporary directory — and the service for which it is native is no longer allowed to read from there.
Then the insidious part begins. In permissive mode, a violation is only logged, so SELinux configuration goes “successfully”: the engineer sees a green status, and after switching to enforcing the service starts and serves only part of the requests.
Denials go to the audit log of violations, not to the application log, so the failure is sought anywhere except the real cause.
The cost of this picture is not a single outage, but gradual degradation. Administrators either disable mandatory access control entirely, returning to the original risk level, or add permissions at random.
A month later, no one can explain why the next rule was granted, and policy diagnostics turns into archaeology.
The order of work here is the reverse of the usual one: first audit denials — what was denied access to which object, which labels actually exist on files and ports — and only then edit the policy. Let's break down how this is closed.
Measurable Effect: What Changes After Implementing MAC Policies
After implementing mandatory access control SELinux or AppArmor, not only the permission model changes, but also measurable operational metrics.
The policy describes what a specific process is allowed to do, regardless of the account — therefore the effect is visible in incident metrics, not only in the audit report.
Attack surface is reduced
Permission separation moves to the kernel level: a service sees only its own directories, ports, and sockets; everything else does not exist for it.
A process with a compromised web application no longer reads other people's configurations and does not write to system directories. The class of scenarios “entered through a service — established persistence on the host” stops working at this step.
Incident response time is reduced
Previously, investigation began with the question “what actually happened”: permissions, change history, and open ports were compared. Now a denial is logged with the process name, path, and operation type — the entry point and culprit are visible immediately.
The investigation fits into a single pass through the log, without collecting all events from the entire host.
Service predictability
The policy lists the standard set of operations, so the service behaves the same on staging and production: after an update, it does not accidentally gain expanded permissions.
Deviations from the allowed set are clearly visible, rather than surfacing weeks later as strange errors.
A class of failures eliminated
Starting under a different account, inherited permissions after manual editing, a script from an unrelated directory — these situations are closed by policy before they appear. The release acceptance checklist shrinks: some guarantees are provided by the kernel, not by procedure.
Protection Modes and Configuration Methods: Comparing Options
The choice of mode determines what happens when a process deviates from allowed behavior: the event is only logged or the operation is blocked. For SELinux, configuration begins with this decision — the implementation order and the scope of analyzing triggers depend on it.
What to choose
| Variant | When it fits | Limitations |
|---|---|---|
| Permissive observation mode | First run of rules on a production node, inventory of accesses | Blocks nothing: violations accumulate in the log, and there is no protection as such yet |
| Enforcing mode | Services with a stable set of operations, public nodes | False positives after application updates — analysis and rule edits are needed |
| Profiles for individual applications | One or two services with known behavior, process isolation | Cover only their own process; the rest live outside mandatory restrictions |
| Complete abandonment of mandatory control | Narrow tasks where policy maintenance is more expensive than the risk | Separation comes down to ordinary file system permissions |
Working order — start in permissive observation mode, collect the log for a full service operation cycle together with updates, then switch the node to enforcing mode.
Exceptions are left only where process behavior has been analyzed: an unexplained rule nullifies the meaning of the access policy.
The limitation of the approach is that access policies require maintenance.
A service update changes the set of accesses, and without a rule owner, mandatory control becomes a source of denials — profiles for individual applications narrow the risk area but do not replace isolation and regular updates.
Policy Implementation Order: From Audit to Handover
The order is structured so that no production service stops because of policy: first we collect facts, then we restrict. Skipping any step returns us to manual analysis of denials at the moment when the service is already unavailable.
- Inventory and configuration audit. Record lists of services listening on ports, their data directories, sockets, exchange directories, and mount points. Separately note where local exceptions already exist — they will have to be rebuilt.
- Observation mode. Switch the environment to a gentle mode where violations are only logged, not blocked, and keep it for at least a full cycle: daily jobs, log rotation, package updates, nightly export.
- Collecting and analyzing triggers. Group audit records by process and access type, separate standard behavior from actual exits beyond the profile. At this step it is clear which services will need to be described more precisely and which write to unexpected locations.
- Formalizing rules. A separate module for each service instead of one common module: when a service is updated, only its profile is rebuilt, not the entire environment. We describe exceptions precisely, tied to a path and label.
- Testing on a copy of the environment. Bring up a snapshot, enable blocking, and run regression: startup, migrations, backups, node failover.
- Commissioning. Enable by service groups, with an observation window after each group and a ready rollback to gentle mode.
- Handover. Policies and the regulation for updating them go into the repository: dependency diagrams, exception log, and the procedure for adding a new service.
Configuring SELinux: Configurations, Policies, Documentation
The result of implementation is not only a working server, but also a package your team can use to maintain the policy without us. Below is the list of what remains after handover.
- Configuration files — the state of the main config and boot parameters captured from the production server, with notes on which values are mandatory and which may be changed.
- Policy modules — sources and built modules for each service, with a comment explaining what the domain is responsible for and why the rule is formulated exactly that way.
- Access flow diagram — a map: which process runs in which domain, which file types and network ports it accesses, and where the prohibition is.
- Change procedure — the sequence: collect a report on blocks on the test environment, form a module, verify it, and only then roll it out to production.
- Instructions for the on-call shift — what to look for in the audit log, how to distinguish a false block from an exploitation attempt, and how to narrowly relax a rule without removing protection entirely.
- Rollback procedure — a snapshot of configuration and policies before changes plus the return procedure if a service update breaks existing rules.
- Coverage map — the list of services under policy, the list in progress, and an explicit indication of what was intentionally left unrestricted and why.
Case: Service Stopped Failing After Enabling Restrictions
The project began with incident analysis: a service on a Linux host went down several times per quarter, and each time the cause turned out to be a process that had gained extra permissions after manual intervention.
SELinux was enabled, but in the mode of logging violations without blocking — the kernel logged them and let them through, so records accumulated in the logs, and there was no real boundary for the application.
What the metrics showed before
The downtime share remained at several hours per quarter: the average episode was about 50 minutes, and recovery took from 40 minutes because the engineer manually matched kernel records with the configuration.
More than two hundred open allow rules had accumulated — half were added “just in case” during another failure. Repeat incidents returned every two to three weeks, and edits did not reduce their number.
What was changed and what was achieved
We rebuilt the policy from scratch: we left only permissions confirmed by tracing production scenarios, and switched domains to enforcing mode one by one. The number of open rules dropped to about forty, each described and tied to a specific process or directory.
Recovery time after a failure fell to a few minutes: the policy layout can be read by a human, not only by the kernel.
Over two quarters of observation, there were no repeat incidents, and the downtime share decreased severalfold. The AppArmor profile on the second host was built using the same scheme — from tracing real calls, not from denying everything indiscriminately.
How Permissive Mode Differs from Enforcing
Both modes run on the same loaded policy — they differ only in whether the verdict is enforced.
In permissive observation mode, the rule is checked on every access to a file, socket, or process, but blocking does not occur: the application gets access, and the system only logs the violation.
Enforcing mode stops the operation on the same verdict — the process receives a denial and cannot retry.
Hence the main risk of transition: a set of permissions that looked workable in observation begins to break familiar scenarios under blocking.
The same principle applies in AppArmor, where the analogue of observation is complain mode: the profile is active and logs triggers, but does not deny access.
The purpose of the intermediate step is exactly this — to see system behavior under live load before prohibitions take effect.
The trigger log is the only reliable source for switching between modes. Denials are extracted from it with the domain, target type, and object class: they show which process is knocking in the wrong place and where the policy is simply too narrow.
First false positives are separated from real denials, unnecessary restrictions are removed, the result is checked on the test environment — and only then is enforcement enabled.
A safe transition is done in stages: the global mode is left permissive, and domains or services are switched to enforcing one by one.
If a service does not come up after enabling, returning to observation takes seconds and does not require a reboot — only the specific domain gets the denial, not the entire node.
Enforcing mode is justified when the log is clean of unexpected triggers and there is a rehearsed rollback procedure.
Common Technical Doubts Before Implementing Policies
Doubts here are natural: mandatory control is a layer that can block application operation where ordinary access permissions allowed it.
Therefore we build implementation not as “enable and see,” but as a controlled transition across nodes and services with observability at every step.
A non-standard application will not work under enforcing — what then?
First the node is switched to permissive mode: policies are applied but do not block, and all denials go to the log with process and file labels.
The engineer analyzes them and describes a local module or rules for the necessary paths and types — edits are precise, not “allow everything.” We switch a specific service to enforcing, not the entire machine, so as not to drag the rest along.
What happens during system and application updates?
The policy relies on paths, labels, and types, not on a specific build, so an update usually does not break it. Risk arises when the directory structure changes or a new auxiliary process appears — then denials are visible in the log again.
We run the update on a test stand with real load and compare the log before moving it to production nodes.
Is downtime needed, and how long?
For most services, downtime is not required: policy deployment and denial collection happen on a running system in permissive.
A short stop is possible only when switching to enforcing for services that cannot be restarted on the fly — we agree on the window in advance and keep it within one maintenance cycle.
Who maintains policies after handover?
We transfer modules, configurations, and the exception list into a version control system with comments for each rule. Along with this comes an instruction for typical scenarios: a service was added, a data directory was changed, denials appeared after an update. Your engineers make such edits using a checklist; we get involved in non-standard cases.
What risks remain, and how to roll back?
A policy that is too narrow breaks operation in a rare scenario; one that is too broad provides no protection. We verify the balance on the test stand, not in production. Returning to permissive is one command; the node continues serving load.
We separately record pre-implementation states so that rollback takes minutes, not hours of analysis.
Let's Discuss the Task and Return with an Estimate and Work Plan
For a substantive discussion of the task, source data is needed: without it, timeline estimates turn into guesses, and access restriction policies are written “from memory” and break production services on the first update. Send what you already have — configurations, exports, diagrams, free-form notes.
- Composition of services and nodes: which applications run, under which accounts, which directories, ports, and sockets they actually need.
- Environment diagram: where the boundaries run between the perimeter, application, and data, which nodes exchange traffic.
- Current protection state: whether permission restriction mode is enabled or only audit mode so far, which profiles are already applied, what has been added manually.
- Constraints and prohibitions: what cannot be changed — update procedure, allowable downtime, maintenance windows, availability requirements.
- Access goals: who needs to access which data, which operations must be allowed, and which must be blocked unconditionally.
- Roles and automation: which employees maintain what, which tasks are performed by scripts and under whose name.
- Logs and alerts: where events are collected, whether there is control over blocks and service crashes after a policy change.
The next step — we review what you send, clarify what is missing, and return with a work plan: staged transition of policies to restriction mode, rollback scheme, and timeline estimate for each environment. Some data can be collected directly on the servers and included in the plan as a separate stage.