Rootless containers on Podman: what breaks in the familiar scheme
The classic container launch model assumes that the control daemon runs as superuser, and the process inside gets root privileges by default. While everything is stable, this goes unnoticed: the service lives in its own isolation, volumes are mounted, ports are published.
But should a vulnerability or permissions error surface, the attacker ends up not in a sandbox but on the node itself, with a view of neighboring containers, the control socket, and the host filesystem.
The price of such an architecture is a single point of failure. One compromised container equals compromise of the entire machine: secrets of neighboring services are extracted from it, shared volumes are read, configurations are edited. Service isolation ceases to be a security boundary and remains only convenient packaging.
Podman removes the privileged daemon from the equation: in rootless mode, containers run in a separate user namespace, where root inside corresponds to an unprivileged identifier outside.
Even with a successful breach, the application gets the rights of an ordinary user — without writing to system directories and without access to other container infrastructure.
The transition changes habits: volumes and ports below 1024 are configured differently, some images require explicit ID mapping, and management without a system daemon relies on other mechanisms.
We'll break down how this is addressed and what exactly needs to be rebuilt in the existing environment.
What migrating services to containers without root privileges provides
Migrating services to containers launched by an unprivileged user changes not convenience but the class of failures and blast radius. A process without root privileges will not mount host directories or write to system paths — container compromise remains within its environment.
The attack surface is reduced not by individual settings but by the launch model itself: the service does not have rights to what is usually exploited.
Next is failure isolation. An error in one service no longer means the whole node goes down: each container has its own memory and CPU limits, and resource exhaustion by one process does not take the others with it.
The failure class "the node crashed — all services went down" leaves practice, along with it the most expensive part of service downtime — the time to bring up the entire host.
Container autostart is described in advance: services come up after reboot in a specified order, with restart policy and dependencies among themselves.
Service recovery time is measured in tens of seconds instead of manually figuring out "what to start and in what order" — the engineer is not occupied with routine but only checks the result.
And a unified development and production environment: locally and on the server, the same image and the same launch command are used, only environment parameters and mount points differ.
This removes discrepancies of "it works on my machine, not on the server," and Podman on the workstation behaves the same as on the production node, so pre-checking ceases to be a separate stage.
Podman and alternatives: comparison of launch modes
The launch mode determines what happens if a container is compromised and how predictably the service will survive a node reboot. A privileged engine is convenient: the container sees host devices, occupies a port below 1024, and mounts any directory.
The price is a shared daemon and a process with root privileges, where an error in the image turns into full access to the node.
A container running as a service user removes this limitation: the kernel keeps the process within that user's privileges, and going beyond them requires a separate configuration error.
Compatibility with familiar commands is preserved — Podman repeats the launch syntax, so deployment scripts are transferred by replacing the invocation, not by rewriting.
The limitations of the mode are visible in practice: privileged ports require forwarding, layers are written in user space, and a dedicated subuid/subgid range is needed.
| Option | When suitable | Limitations |
|---|---|---|
| Privileged engine per node | Access to devices, host volumes, ports below 1024; inherited images | Shared daemon — single point of failure; a container with root inside equals root on the node |
| Container as a service user | Web services, queues, builders, background tasks without special privileges | No ports below 1024 without forwarding; slower writes to layers; some kernel capabilities unavailable |
| Native services outside containers | Single process, updates via package manager, isolation not needed | Dependency on host libraries; no portability; version rollback is harder |
A mixed scheme is workable: rootless by default on all nodes, individual machines with a privileged engine for specific workloads. The key is to record which privileges are actually needed and not grant more.
Service migration procedure: from node audit to handover
We migrate services to rootless containers (Podman) in stages: each next one relies on the result of the previous. An error is visible at the step where it appeared, not at the moment a production service fails.
- Node audit. We collect an inventory: which services are running, under whose identity, which ports and directories they occupy, what starts from systemd and the scheduler. We record dependencies between services — without them, the placement scheme will be arbitrary.
- Service placement scheme. We distribute services across users: what goes into the rootless environment, what remains system-level, where shared volumes and port forwarding are needed. We agree the scheme with you before making changes on the node.
- Environment preparation. We enable user namespaces, set resource limits and rules for long-running sessions. We check that containers live after the administrator logs out — otherwise they go down with the session.
- Isolation and autostart configuration. Each service gets its own user, its own volumes and ports, and file permissions only where needed. We implement autostart using service descriptions with an explicit startup order that mirrors real service relationships.
- Fault tolerance testing. We reboot the node, cut the network, shut down services one by one, fill the disk. We see what comes up by itself, what loses data, and what requires manual intervention.
- Handover to operations. We provide the configuration, placement scheme, instructions for standard operations and rollback. We run support through the scenarios so that node operation does not depend on the implementation author.
Handover package: configurations, schemes, access regulations
The handover is structured so that the customer's team is left with a working environment, not verbal agreements.
Podman's rootless mode adds its own nuances: user permissions, ID mapping, separate namespaces — everything that is easy to break during the next change if it is not documented.
Therefore, we hand over a set of artifacts in a repository that undergoes review like code and is updated together with the service.
- Configuration files. Units for running containers as user services, registry and image storage settings, resource limits, log rotation parameters — with comments on what changes and why as load grows.
- Deployment schemes. Placement of services across nodes and user accounts, which volumes and network namespaces are shared, where the boundary between development and production environments lies.
- Access policies. Which employees and groups manage services, from where image builds are allowed, signature verification rules, prohibition of privileged containers and mounting system directories.
- Update regulations. The procedure for rolling out a new build, checks before switching traffic, behavior with busy volumes, approval points, and criteria by which a release is considered successful.
- Documentation and rollback. Step-by-step instructions for standard operations, a service dependency diagram, the procedure for returning to the previous build with verification that the service came up and responds to requests.
- Diagnostics and knowledge transfer. A set of commands for incident analysis, review of configurations on a live environment, and answers to the customer's engineers' questions — so that support does not depend on us.
Case: migrating services to containers without privileges
Services on 42 nodes ran as superuser and shared access to the container engine socket. Any compromised component got host privileges, and a failure of one service was often fixed by restarting the entire machine.
Recovery after such a failure took 35–40 minutes, updates were moved to a 3-hour night window, and some rollouts were rolled back manually. The failure class "restart the entire machine" occurred on average 9 times per quarter.
What was changed
Services were moved to Podman rootless mode: each runs under a separate system user without access to the privileged engine. Autostart uses user systemd units with lingering, so the service comes up after a node reboot without manual login.
Volumes were connected as bind mounts with SELinux labels, port publication was limited to the required list, and images came from an internal registry with signature verification. The limits policy and logging were prescribed at the service level, not the node level.
What became
| Metric | Before | After |
|---|---|---|
| Share of nodes without superuser privileges | 0 of 42 | 38 of 42 |
| Service recovery after failure | 35–40 min | 4–6 min |
| Failures of the "restart the entire machine" class | 9 per quarter | 0 |
| Update window | 3 h at night | 25 min during the day |
"Previously, one failed service dragged the entire node down with it; now only its process tree restarts," — support engineer.
The second participant is shorter: "The update window is no longer at night — we roll out during the day and don't wake the on-call engineer."
Four nodes were left with privileges: the services there need direct access to hardware, and this is recorded by a separate policy. The rest is closed — process compromise no longer grants host privileges, and recovery time has been reduced by almost an order of magnitude.
What to do if Podman is already partially deployed
While Podman covers some services and the rest run under the old scheme, this is a working state, not an emergency. The danger is not in the coexistence of two launch models, but in dual maintenance: two sets of access rules, two ways to view logs, two ways to update images.
The task is not to roll back what has been done, but to set the order in which the remainder migrates.
The first step is inventory: what is already in rootless mode, what remains, and what each service depends on — volumes, ports, accounts, names of network neighbors.
Most stories of "it broke after migration" are explained precisely by dependencies: a neighboring service does not resolve by name in the new scheme or a directory becomes inaccessible due to permissions. This is recorded before any moves.
Next is the priority queue. Services without their own state migrate first: external proxy, cache, queue handlers — they come up in seconds, and an error costs minutes.
Services with data go last: they need a planned window, a consistent copy, and integrity verification afterward.
While both schemes are alive, ingress and networking are moved to a level independent of the launch method. Names and entry points remain the same, and services do not know where a neighbor lives — the transition stretches without stopping operations.
It is completed not when everything is running in rootless mode, but when each service has one launch method, one log delivery method, and one update method. Dual maintenance ends with the last duplicate set of rules.
Isolation, autostart, and compatibility: technical concerns
All objections to Podman come down to four or five scenarios: how to deliver traffic to low ports, what about data permissions, will ready-made images run, will containers survive a node reboot, and can services be migrated one by one.
We address each in substance — indicating where configuration is needed and where a limitation truly exists.
What to do about privileged network ports?
A rootless container cannot occupy a port below 1024 by itself. There are two workable approaches: allow unprivileged processes to use low ports on the node, or keep a reverse proxy or load balancer outside that accepts traffic on 80/443 and forwards it inside to a high port.
The second option is preferable: the container does not get extra privileges, and there is a single traffic entry point managed by a separate service.
How is access to the filesystem and network resources handled?
Data directories are mounted with an explicit owner — the user under which the container runs — otherwise the process inside cannot write.
On nodes with SELinux, volumes are labeled with context labels during mounting, otherwise access is blocked at the mandatory access control level.
Internal networks are created as user networks, services see each other by name, and only the required ports are published externally.
Are existing images compatible?
Podman works with OCI images, so previously built images run without rebuilding — the launch command and the way services are described change. The main edits concern descriptions: mount paths, network names, healthcheck, and dependency startup order.
These discrepancies are identified during migration in a test environment, not on the production node.
Will containers come up after a node reboot?
By themselves — no: rootless processes are not brought up by the system init manager. This is solved with user units and enabled lingering; then services start at boot, restart after failure, and stop correctly in the specified order. This also provides a clear picture of dependencies and logs.
Can some services be migrated and some left as they are?
Yes, and this is the safest path. Nodes with different environments coexist as long as they do not conflict over ports and volumes; an external proxy routes traffic by service.
Migration proceeds in groups, with rollback for each group separately — a failure on one service does not affect the others.
Let's discuss the task: what data is needed for an assessment
A substantive assessment is assembled from facts about the infrastructure, not general wishes. The more accurately you describe the current state and constraints, the fewer assumptions in the work plan and the more realistic the timelines. Below is the minimum that is usually enough for us to return with an assessment and plan.
- List of nodes and services. How many hosts and what their roles are, what is already in containers and what will remain on the host: system daemons, task scheduler, local databases, file services.
- Current launch method. Compose files, systemd units, deployment scripts, your own image registry — they show what is transferred as is and what requires repackaging.
- Isolation requirements. Whether separate users and network segments between services are needed, access to privileged ports, shared volumes — the rootless launch scheme in Podman and the order of granting permissions depend on this.
- Update windows. When services may be restarted, what downtime you consider acceptable, whether there are strict unavailability periods, and in what order nodes can be rolled out.
- Data access. Where volumes and configs are located: local disks or network storage, what is mounted, how backups are arranged, whether there is data that must not be moved outside the node.
- Target metrics. What we consider the result: node bring-up time from scratch, recovery time after failure, share of services without root, share of automated checks.
Describe the task in free form — that is enough for us to return with an assessment and work plan.