Elixir development: where the backend hits the ceiling
As long as there are hundreds of concurrent connections, the backend responds steadily.
At thousands, the picture breaks down: response time grows nonlinearly, some requests never reach the handler, and clients resend them — so instead of one request in the system, two or three copies appear.
Restarting a node finishes off live sessions: the user returns to an empty form, and support returns to a stream of identical inquiries.
The cause is rarely hardware. The backend bottleneck is the execution model: if every request occupies a separate thread for the entire time it waits on the database or an external service, concurrency is capped by pool size.
The queue grows faster than the load, and latency under load sustains itself: the longer the response, the more retries.
Elixir development removes this ceiling. Virtual machine processes are lightweight — hundreds of thousands can be held simultaneously, and the scheduler distributes work across cores without manual pool tuning.
A handler does not block while waiting: it suspends and yields resources to others, so response time grows linearly rather than collapsing.
The second effect is supervisors. The node, subsystems, and connections are described as a tree: a crash of one process restarts only that process, not the entire application.
A version update proceeds without dropping connections; session state survives a restart — a lost connection no longer means lost user work.
We start with the load profile: exactly where your high-load backend hits the ceiling, which requests are duplicated, and what happens when a node fails. Then comes a transition plan, migration order, and estimate.
Resilience under load: connections, queues, retries
Under load, it is not the code that breaks first but the links between services: the connection pool is exhausted, the inbound queue grows, and retries finish off an already overloaded node.
Each such failure looks different, but the cost is the same — growing latency and a cascade of timeouts. We break down what provides resilience under load and how to measure it.
Failure isolation is built on a process tree: one connection is a separate lightweight process with its own state. A handler crash does not bring down the node; supervision restarts only the failed branch, while the rest continue serving traffic.
This is where the difference in service recovery time appears: seconds to bring a branch back up instead of minutes to restart the whole application.
Connection retention is solved not by growing pools but by limiting them and deliberately degrading. A node holds tens of thousands of concurrent connections without linear latency growth because waiting does not occupy a separate thread.
When the queue limit is reached, the service returns a reduced result or cached data instead of accumulating requests until timeout.
We make request retries idempotent: one operation key, increasing backoff, random jitter, and a cap on the number of attempts. This eliminates three classes of failures — pool exhaustion, cascading timeouts, and a thundering herd of synchronous retries that amplifies the peak.
In Elixir, these mechanisms are at the foundation of the platform rather than bolted on top of the application.
We measure the result using three metrics: tail latency at peak, the share of rejected requests, and time to full recovery after a single node fails. Their dynamics before and after the changes show real resilience — unlike promises that it will “withstand any load.”
Implementation options: persistent processes, hybrid, ready-made services
The execution model determines where connection state lives and what happens when a node fails.
In Elixir development, persistent processes hold each session’s state in isolation: one crash does not drag down the others, and the scheduler distributes load across cores without manual locks.
A classic stack with shared mutable state is easier to hire for, but as the number of concurrent connections grows, it hits locks and shared cache coordination.
| Option | When it fits | Limitations |
|---|---|---|
| Persistent processes | Many concurrent connections, long sessions, streaming, chats, telemetry | State cannot be kept in shared storage; the team must learn the isolation model; restarting live state requires a well-designed scenario |
| Classic stack with shared state | Steady CRUD profile, rare long-lived connections, team already owns the stack | Locks and shared cache become a bottleneck; as connections grow, instance count and synchronization overhead grow |
| Hybrid | The core stays on the current stack, real-time mode is moved to a separate environment | Two environments in operation: shared authentication, unified observability, data consistency between them |
| Ready-made cloud services | Need fast event exchange, no in-house team for it, predictable load | Billing by number of connections and messages, data format restrictions, vendor lock-in, migrating in-house is difficult |
A hybrid is chosen when there is no point in rewriting the core, but live connections are already hurting: the main flow stays on the previous stack, and events go through a separate process-based environment.
The approach is not suitable if data must not leave your environment, the load profile is bursty, or full control over latency is required — then we consider persistent processes as the main option.
How Elixir development proceeds: stages of engineering work
Our work order is the same for a new service and for migrating an existing one to Elixir — only the scope changes. The implementation stages are structured so that every architectural decision relies on a measured load profile, not assumptions about it.
- Load audit. We capture the profile: the share of read and write operations, request duration in different areas, behavior during peak hours, and limits of external systems. The output is a map of bottlenecks and a list of what must survive failure.
- Process distribution scheme. We split the domain into isolated processes under supervisors: where to keep state in memory, where in shared storage, and where to access the database directly. This way, a failure in one area does not drag down the others.
- Cluster and queues. We configure a distributed cluster and background processing: retries, concurrency limits, protection against duplicate jobs, and message ordering.
- Failure testing. We deliberately take a node, a link, and an external dependency out of service, and see whether the service recovers on its own, how many requests are lost, and how quickly service is restored.
- Acceptance by metrics. We compare the result against thresholds agreed before the start: share of successful requests, peak response time, memory consumption, and background queue processing speed.
- Handover to operations. We provide documentation, supervision schemes, on-call regulations, configured metrics and alerts, and train the customer’s team.
What we hand over: schemes, regulations, and documentation
Project handover is not an archive with source code, but a package that lets an on-call engineer bring up the environment, find the cause of degradation, and restore the service without the code’s author.
We verify completeness in practice: we deploy a stand strictly according to the documentation and record where we had to guess.
- Environment configurations: cluster parameters, connection limits, timeouts, replication, secrets handling — with an explanation of why each value is what it is.
- Data flow schemes: the request path from the load balancer to storage, queues, background jobs, and places where an event can be lost or duplicated.
- Fault-tolerance schemes: behavior when a node fails, the database is lost, or an external service is unavailable; switchover order and time to return to service.
- Access policies and roles: a role–action–resource matrix, rights to logs, configuration changes, access to the production environment, and key rotation.
- Backend documentation: module structure, interface contracts, data migrations, behavior on redelivery, and load limits.
- Operations regulations: on-call, response to monitoring signals, escalation, release and rollback procedure, and planned maintenance.
- Recovery plan: typical incident scenarios, backup points, target recovery time, and verification through drills.
Case: fivefold peak without drops and lost requests
The service received a stream of requests from several channels — a website form, a mobile client, and partner integrations.
At peak, connections dropped: the client did not receive confirmation of receipt and sent the request again, while some inquiries were lost between receipt and recording.
The share of lost requests reached several percent, response stretched from tenths of a second to several seconds, and operators resolved discrepancies manually.
Receipt was separated from processing: the request is recorded in a log and confirmed to the client immediately, then a handler picks it up, and the burst is smoothed by a rate-limited queue. A resend is recognized by the idempotency key and does not create a duplicate.
Fault tolerance was built on process supervision: a failed processing node restarts on its own, and the request state is restored from the log.
The core was reworked in Elixir — the lightweight process model holds tens of thousands of concurrent connections without latency growth.
What changed at peak
| Metric | Before | After |
|---|---|---|
| Lost requests | several percent per day at peak | 0 — the inquiry is confirmed and stored |
| Response latency | seconds, grows with load | fractions of a second, independent of peak |
| Recovery after node failure | tens of minutes, manually | tens of seconds, automatically |
Incident postmortems are no longer a constant activity: operators work on requests rather than reconciling missing inquiries.
What to do if part of the system already runs on another stack?
Yes — part of the system can be moved to Elixir without stopping the product, but the boundary is drawn not by layers but by data ownership. First we establish which entity belongs to whom: every table and every write scenario must have exactly one owning service.
Others access it through a contract, not its database directly. Without this condition, a mixed stack quickly turns into data divergence that takes weeks to trace.
We place the boundary where the nature of the load changes: long-lived connections, event streams, notifications, chats, and payment scenarios. We move such a piece entirely — migrating half a domain costs more than the migration itself.
At the junction, we leave a thin compatibility layer that knows both environments and can route calls in either direction.
The gradual load migration proceeds in shares: the gateway sends part of the traffic to the new service and the rest to the old one. First comes shadow mode: both environments process the request, responses are compared, and discrepancies are resolved before the user sees them.
The share is increased based on metrics — error rate, latency, behavior under peak — not by the calendar.
We do not use dual writes at all: only one owner writes, and the second environment learns about the change through an event. For retries and interruptions, we use an idempotency key and a transactional outbox table so an event is not lost during a failure. This way, the two environments do not compete for the same data or overwrite each other’s edits.
We maintain service compatibility through contracts: the request schema is versioned, and we do not remove old fields as long as at least one environment reads them. The rollback switch remains until the new service steadily handles the full load. Then we shut down the old one, migrate the remaining data, and remove the dual routes.
Technical concerns: fault tolerance, hiring, support
Technical concerns around a real-time backend are usually the same: how the system behaves when a node fails, who maintains the code after launch, and where to find engineers for a narrow stack. Below are short answers to what we most often discuss before the start.
How do you test system behavior during failures?
We reproduce failures deliberately: we stop a cluster node under load, break the database connection, reset websocket channels — and see how many sessions reconnect without data loss.
The Elixir supervisor model works: a crashed process is brought back up, and session state is restored from storage. For acceptance, we provide numbers: recovery time and the share of lost messages.
How do you find engineers for this stack?
The market for this branch is narrow, and pretending otherwise is pointless.
The core team is ours, and we expand by moving developers from other functional stacks: the syntax is learned quickly; the longer part is getting used to immutable data and restarts instead of exceptions. The first tasks are off the critical path, with a mentor and mandatory review.
How is support organized after launch?
An on-call shift by schedule, alert channels, and an escalation procedure. We build observability on precursors of failure, not on the fact of a crash: growing message queues, processing latency, and process restart frequency. Changes are released in a low-load window, and a release is rolled back with one command.
Describe your task — we will come back with an estimate, a plan, and a list of checks that we will include in acceptance.Let’s discuss the task: we’ll analyze the load profile and risks
Before giving timelines, we look at the load profile: how many concurrent connections the system handles, how traffic is distributed between regular requests and persistent channels, and where peaks occur.
Without these numbers, the estimate and work plan turn into guesswork, and implementation risks surface on the very first release. To discuss the task concretely, five pieces of information are enough:
- Load profile — peak values of concurrent connections and requests per second, share of real-time traffic, daily and seasonal spikes, response time requirements.
- Current stack — what the backend runs on, the current OTP branch, libraries and message brokers, what has already been migrated, and what still relies on old code.
- Data and storage — schemas, volumes, consistency requirements, hot tables, the backup procedure, and a tested recovery scenario.
- Failure scenarios — what has already failed and how recovery happened, which nodes have no redundancy, where messages are lost, and how quickly the system degrades under overload.
- Infrastructure and processes — where the service is deployed, how releases and rollback are organized, whether observability exists, and who makes priority decisions.
Based on these answers, we assemble a map of implementation risks: where a separate load environment is needed, what is rewritten entirely, and what is migrated in stages.
After that, we come back with an estimate and a work plan — with stages, acceptance criteria, and rollback points.