Configuring Fly.io: why response times lag for some clients
Typical picture: the application is deployed in a single region, and the database and cache live there too.
A user from a remote region goes through transit networks — and pays for this with distance in every request, while at peak some connections break during the handshake and first bytes of the response.
At the same time, metrics often look healthy: average response time is normal because the bulk of requests come from the "home" region and blur the tail.
The bottleneck is visible only in the 95th and 99th percentile and in the geographic breakdown — that is where the failures for some users come from, the ones no one can reproduce in a staging environment.
The second cause is a single point of failure. While the application lives on one site, provider degradation, scheduled maintenance, or hitting a connection limit makes everything unavailable at once.
Scaling within one region removes load but not the distance to the user, so response times for remote clients remain the same.
Configuring Fly.io solves both tasks: application instances are launched in multiple regions, request routing directs traffic to the nearest healthy site, and unhealthy ones are automatically removed from load balancing.
The behavior when a region is lost is described separately — the request does not wait for a timeout but goes to the next site.
We break down how this is covered: from selecting the set of regions and routing rules to failure policy and instance health checks.
What a well-designed application placement scheme provides
A placement scheme is not "enable one more region." It is a decision about where application copies live, how traffic is distributed between them, what counts as node failure, and by what rule load moves to a neighboring site.
A reworked scheme changes four measurable things: response time, recovery time after failure, the share of failed requests, and the class of failures that reach the user at all.
When application copies are placed in regions near users, a request goes to the nearest site rather than halfway around the world — latency stops being a lottery.
The share of failed requests also drops because, when a region fails, traffic stops going there based on health checks rather than on complaints. This is the difference between "some requests fail until we notice" and "node failure is invisible to the client."
Recovery time after node failure no longer depends on manual actions: a copy is launched according to a predefined rule, and load shifts automatically.
This removes an entire class of failures — not "the server went down," but "the server went down, and no one knew about it for half an hour." We measure the cost of downtime not in money, but in the share of sessions and orders that did not survive until a response.
Tools like Fly.io provide placement regions and failover mechanics, but the result is determined not by the fact that they are enabled, but by the combination: which regions are selected, how many copies are kept hot, how health checks are configured, and what is considered sufficient to return a site to service.
Comparison of launch modes: one region or several
The launch mode determines what happens to the application when a zone fails and what latency users from different cities will see.
In Fly.io, it is set not by a single switch but by a combination: how many regions are active, where persistent volumes live, and how traffic is distributed between application machines.
One region is easier to maintain, but all servicing depends on one point — we break down the alternatives.
| Option | When it fits | Limitations |
|---|---|---|
| Single region, one active zone | Internal services, predictable audience, pilot workloads | Zone failure stops service entirely; latency grows with distance to the user |
| Multiple active zones (active mode in several zones) | Public services with broad geography, availability requirements | Data replication and moving state out of application machines are required; the number of observation points grows |
| Hybrid scheme: edge nodes plus your own environment | Exchange with internal systems, requirements for data storage location | Your own environment becomes a single point of failure and a bottleneck for write latency |
| One active zone with manual failover | Infrequent releases, an acceptable downtime window, minimal maintenance | Failover depends on the on-call responder's reaction time; traffic return is checked manually |
In practice, platform limitations show up in data, not code: a persistent volume is tied to a specific zone and does not move itself during migration, so we move the database and files to a replica with a clear rule for choosing the primary node.
We keep sessions outside application machines; otherwise, after failover, the user is logged out. And we check both directions — not only traffic leaving the failed zone, but also returning to it, because return is more often what causes data divergence.
Fly.io configuration procedure: from audit to handover
Fly.io removes part of the infrastructure routine, but distributed placement provides a gain only when regions, request routing rules, and state handling are coordinated with each other.
That is why we follow a fixed procedure: first infrastructure audit and design, then rollout — each change is verifiable at its own step and can be rolled back separately.
- Infrastructure audit. We assemble a map of services and their dependencies, look at where users actually are, and measure current latencies and failure rates. The output is a list of what we migrate and what stays in place.
- Placement design. We determine how many regions are actually needed, how many application instances to keep in each, and where the database, cache, and queues live. We fix placement parameters and autoscaling rules in configuration in the repository, not in the panel.
- Routing and network rules. We configure bringing requests closer to the nearest region, health checks, behavior when a region is unavailable, and connection limits. We test on a staging environment with synthetic traffic, not on live users.
- State and migrations. We place database replicas next to the application, prepare a zero-downtime migration scheme, and ensure backward compatibility during rollout.
- Load testing. We run a real traffic profile with a 3x spike, forcefully shut down a region, and restart instances. We look at recovery time and the share of successful requests.
- Release and rollback. Canary rollout to some instances, criteria for automatic return to the previous image, and analysis of false positives.
- Handover to operations. We hand over access, alerts, diagrams, and the incident response procedure, train the team, and jointly review the first incident.
Each stage is closed with an artifact: configuration in the repository, placement diagram, test report, instructions. Handover is considered complete when your team can launch a region and roll back a release on its own without our help.
What stays with you: configurations, diagrams, procedures
We do not leave you with verbal agreements and panel access: everything configured is described in text and stored in your repository.
An engineer who comes after us — in-house or third-party — deploys the environment from these files, without correspondence with us and without reverse-engineering the console from logs.
- Configuration files in the repository. Description of the application on Fly.io, machines, regions, availability checks, and environment variables — in a form that passes review and is rolled out with one command, not manually in the panel.
- Infrastructure diagrams. Connections between applications, databases, cache, queues, and external services: what depends on what, where entry points are, and what degrades together with what.
- Access policies. Roles and tokens, boundaries between environments, who can change what in production, where secrets are stored, how they are rotated, and what to do in case of a leak.
- Operations documentation. Where to look during degradation, where metrics and logs are, what alerts are set up and what to do for each, and how to perform typical operations — scaling, adding a region, switching traffic.
- Update and rollback procedure. Rollout order, stop criteria, the procedure for returning to the previous build, and the time it takes to perform such a return.
- Acceptance checklist and access handover. What to check before acceptance, what rights are issued to you, and what is revoked from us.
Administrative rights and keys are also an artifact: we record their transfer to you rather than keeping them ourselves "just in case." After acceptance, the infrastructure operates and evolves without our involvement.
Implementation case study: metrics before and after
The first version of the service was deployed in one region: the database and application lived in the same cluster, and users came from a dozen countries.
At peak, when daily load tripled, p95 response time went beyond one and a half seconds, the share of error responses reached 4%, and a failed node was raised manually — about forty minutes of downtime per incident.
What was changed in the scheme
The application was distributed to edge nodes near users, and routing was switched to selecting the nearest site with automatic traffic shifting when a neighboring one degrades.
State was moved to external replicated storage, so nodes remained interchangeable: any machine can be lost without data loss.
Fly.io provided machine management through a declarative config: the number of instances is described in a file, and scaling is tied to metrics.
Liveness checks and auto-restart replaced manual on-call duties — a failed node is removed from rotation and brought back up without an engineer's involvement. Scheme edits go through the same config, so environment configuration no longer diverges between staging and production.
Metrics before and after
| Metric | Before | After |
|---|---|---|
| p95 response at peak | 1.4 s | 300–400 ms |
| Share of error responses | 3–4% | 0.3% |
| Node recovery time | ~40 min | 20–30 s |
| Load headroom at peak | queue grew | handles three times more requests |
Node recovery time decreased because degradation is determined by checks, not by user complaints, and traffic moves to a neighboring site before full recovery.
The error rate dropped not only because of moving closer to users, but also because failure of one node stopped being a service failure.
How launching in one region differs from several
The answer depends on what exactly you are optimizing: user response time or ease of operations.
On one site, all requests are served by one application node and one database — latency is entirely determined by the user's distance from that point, and data consistency comes "for free": writes and reads go to one storage, and there are no races between replicas.
As long as the audience lives in one region, such a scheme is cheaper to support and easier to debug: one set of logs, one scheme, one recovery scenario.
With the appearance of a second region, what is added to latency is not speed but coordination.
Fly.io directs traffic to the nearest site, and this saves time only on reads from a local copy: writes still go to where the primary database is, which means a user from a remote region still pays for them as before.
Node redundancy changes the class of failures. Two sites turn a region outage from downtime into degradation: a liveness check removes the node from rotation, and traffic flows to the neighboring one.
But you need to decide in advance where writes are moved, how much data loss is acceptable, and how the application will behave during failover.
Consistency is the main price of distribution. A user must see their own just-made write, so the session is pinned to a node, retried requests are made idempotent, and schema migrations are applied sequentially on all sites.
If the audience is unevenly distributed, a second site often complicates operations more than it speeds up the user experience.
Common technical doubts before starting
Doubts at this stage are usually the same: what to do with an already running application, how to understand exactly where response time is lost, and what will remain if the platform has to be changed. Below are short, to-the-point answers.
We already have an application deployed — will it have to be rewritten?
Usually not. The application is built into an image and described by a configuration file, so it moves together with its logic; entry points, environment variables, and release rules change.
We first bring up a copy in a test environment, compare behavior, and only then switch production traffic.
How do you distinguish a placement scheme problem from a code problem?
We look at the response time breakdown: if a request takes a long time to reach the application and back, but time inside processing does not grow, the bottleneck is in the route and proximity to the user.
If the handler itself grows, the issue is in code, database queries, or background tasks. We take measurements on both segments and show exactly where milliseconds are lost.
How can we be sure that response time has actually improved?
We compare identical scenarios before and after: time to first byte, the share of requests that meet the target threshold, and behavior under peak load. Measurements are taken from several geographic points so the picture is not averaged. We record the result in a report — before and after, using the same metrics.
What happens if we decide to move to another provider?
The application remains yours: the image, configuration, and release rules are in your repository, without closed-source add-ons. Migration to another provider takes days, not months, and we support it as a separate workstream.
No lock-in to us remains in the architecture — only documentation and access, which we hand over to you.
Let's discuss your task and come back with a work plan
For the estimate and work plan to be based on facts rather than assumptions, we need a minimum of initial data. Gather what you already have at hand: some information is surely in monitoring and the repository, and we will clarify the rest on a call. This is enough to discuss the configuration task substantively and give realistic timelines.
- Current environment diagram: which regions are involved, where the database, cache, and queues live, how traffic reaches the application, and what is already deployed on Fly.io.
- Load profile: peak and average traffic, read-to-write ratio, seasonal spikes and their amplitude.
- Response time requirements: which operations are critical, which countries users come from, what response time is considered acceptable, and what is a cause for concern.
- Constraints: personal data storage rules, environment isolation requirements, acceptable maintenance and update window.
- Current failures: exports from monitoring and logs, incident reviews over recent months, what has already been tried and why the solution did not fit.
- Team and access: who is responsible for infrastructure, what roles are needed in the control panel, and how access to the repository and secrets is organized.
- Growth plans: expected load increase, new regions, tied releases, and hard launch dates.
Next, we review these materials, note gaps and contentious points, and ask substantive clarifying questions. The output is an estimate and work plan: stages, rollout order, responsibility areas, and control points where you see the result.
If some data is missing, that is not critical — we will assemble the picture together and honestly say what can be done now and what requires a separate stage.