Memcached Caching: Where a Web Application Loses Speed
An application that hits the database on every request for the same data runs into a ceiling long before the hardware is exhausted.
A product catalog, a list of categories, user permissions, counters — these queries repeat hundreds of times per minute and are executed from scratch every time.
Response time grows linearly with the number of concurrent visitors, and the database starts spending resources on repetitive operations instead of useful work.
This is especially noticeable during spikes: an advertising mailing or a social media post brings a wave of traffic, and what comfortably handled a dozen users collapses under a hundred.
The queue of database queries piles up, pages respond in seconds, some requests drop off on timeout — while the application server's CPU is only half loaded.
Memcached caching addresses exactly this class of bottlenecks: the result of a heavy query is stored in RAM and served to subsequent requests without hitting the database.
The database stops being the point where all the load converges, and response time stops depending on how many people arrived at once.
The key question is not "whether to enable caching" but what exactly to cache, for how long, and how to invalidate records so that the user doesn't see stale data.
Let's break down how this works and where the line runs between a gain in speed and the risk of serving outdated data.
What Memcached caching delivers in production
Memcached caching addresses the most common class of production degradation: the same queries — catalog, user profile, access rights, results of heavy aggregates — hit the database on every request.
While traffic is steady, this goes unnoticed; during a spike (newsletter, publication, sale), the database hits the connection pool limit, the query queue grows, and response time goes from tens of milliseconds to seconds.
Then come frontend timeouts and failures that look like "the site went down," even though the code hasn't changed.
Memcached here works as a shared layer of ready-made responses in RAM: the application computes the data once, stores it under a key with a time-to-live — and subsequent requests get the result without touching the disk. This delivers three measurable effects.
Response time on hot pages drops severalfold, because fetching from memory doesn't compete for disk operations. The database is offloaded: the share of read requests reaching it is reduced by an order of magnitude, freeing up resources for writes and complex reports.
Resilience to load increases — the application survives a spike on the same hardware instead of falling over, because the cache absorbs the impact.
Separately important is the eliminated class of failures. When the database is overloaded, everything fails at once; when the cache is, degradation is gradual: some requests go to the database, pages respond more slowly but remain available.
This is the difference between "the site is unavailable" and "the site is running with increased latency."
Practice shows: without discipline around keys and invalidation, the cache becomes a source of stale data.
That's why we design the key scheme, reset rules when entities change, and pool separation for different data types — so that Memcached caching delivers a predictable result rather than a new headache.
Cache Operation Modes: Comparing Approaches
The caching mode determines not the response speed, but how the application survives data changes.
The same Memcached in "read-only" mode gives a predictable load on the database, while in "write-through" cache mode with event-based invalidation — fresh data without manual purges. Below are working options that are realistically chosen between at the start.
How to Read the Table
The "when it fits" column describes the load profile under which the mode pays off. "Limitations" — the class of failures you accept along with the choice: stale data, extra requests to the database, or the complexity of warming up after a restart.
| Option | When it fits | Limitations |
|---|---|---|
| Read-only cache | Catalogs, reference data, rarely changing content | Edits are visible with a delay until the key's time to live expires |
| Write-through cache with event-based invalidation | Personal accounts, shopping carts, data with frequent edits | Discipline is required: a missed invalidation = a stale response |
| Distributed cache across multiple nodes | High load, horizontal scaling | Key consistency between nodes, increased response time on a miss |
| Cache warm-up after deployment | Heavy queries, expensive aggregates | A separate deployment stage, otherwise the first requests hit the database |
What This Gives in Practice
Event-based cache invalidation removes the main pain point — the desync between the database and the cache. Cache warm-up after deployment eliminates the spike in requests to the database in the first minutes.
A distributed cache makes it possible to survive a node failure without complete degradation: keys are redistributed, and the application keeps responding.
We fix the choice of mode in the configuration and describe it in the documentation — so that the next engineer doesn't have to guess why the key lives exactly that long and what happens on a miss.
How we implement Memcached caching: stages of work
Implementing caching starts not with installing Memcached, but with an audit of the application: we look at which database queries are executed most often, where response time grows under load, and which queries are repeated without any data changes.
Based on the audit results, it becomes clear what is actually worth caching and what is useless or dangerous to put into memory.
- Application audit and profiling. We build a picture based on real traffic: hot pages, heavy queries, the share of repeated requests, behavior under peak load. The output is a list of caching candidates with the expected effect for each one.
- Cache schema design. We define keys, storage structure, record lifetimes, and invalidation rules: what is reset when an entity is updated, what lives on a timer, and what is not cached at all. Separately, we record which data is critical with respect to staleness — the cache does not touch it.
- Memcached configuration. We deploy the service, set the amount of memory for the cache, the record eviction policy, and connection limits. We select the configuration based on the load profile, not the defaults — the hit rate depends on this.
- Integration with the application. We connect the client, wrap the selected sections of code, and add handling for cache unavailability: if there is a failure, the application continues working with the database instead of crashing.
- Cache verification. We look at the hit rate, response time before and after, and behavior during warm-up and reset. We separately check invalidation scenarios — that the user does not receive stale data.
- Load testing. We run peak scenarios and check how the system behaves when memory overflows and the cache node restarts.
- Handover and documentation. We provide the configurations, key schema, invalidation rules, and operating instructions — so that support can work with the cache without us.
Each stage ends with a verifiable result: before moving on, we record the metrics and make sure the changes have not broken the application's behavior.
Access policies and cache keys: what we configure
A cache lives exactly as long as the agreement about who writes to it and by which key. That is why the delivery includes not only the configuration but also a set of artifacts that the operations team can work with going forward without us.
- Memcached configuration — pool size for the working data set, connection limits, timeouts, moved into a separate environment file rather than hardcoded into the application code.
- Cache key schema — a single template of the form
entity:identifier:version, so that keys do not overlap between modules and do not break when a new version of the logic is deployed. - Access policies — which services and nodes can see the cache, by what attribute they authenticate, what is closed off from the external network; separately, we record who has access to the full flush operation.
- Invalidation rules — what happens when a record is updated: targeted key deletion, a version bump in the name, or a short time-to-live for rapidly changing data.
- Placement schema — where the cache sits relative to the application and the database, how keys are distributed across nodes, and what happens when one of them fails.
- Metrics and thresholds — hit ratio, eviction volume, memory consumption; these show when the cache has stopped helping and needs to be revisited.
- Documentation and procedures — how to bring up the cache from scratch, how to safely flush it in production, and whom to contact when data diverges between the cache and the database.
We agree on each item with your team before rollout: keys and access policies are the places where a mistake quietly corrupts data instead of failing with an error.
After the artifacts are handed over, your engineers can change time-to-live values and add new entities without coming back to us for every edit.
How is Memcached different from Redis for caching?
Memcached is a pure in-memory cache: it stores key-value pairs without data types, without disk, and without replication.
The model is simple: multiple threads serve requests in parallel, memory is divided into fixed-size pages, and eviction follows the "last used — first to be removed" principle.
Hence its strengths — predictable read latency and linear throughput growth when adding nodes: the client itself distributes keys across servers via hashing, and the cluster scales without rebuilding data.
Redis went further: in addition to strings, it holds lists, sets, sorted sets, hashes, and streams, can persist state to disk, replicate nodes, and survive a restart without completely losing the cache.
This changes the role of the solution — from a read accelerator it turns into a component you can rely on in application logic: counters, task queues, rate limiting, locks.
The price for flexibility is more complex configuration, higher memory requirements for the same data volume, and care when choosing an eviction policy so that hot keys are not pushed out by cold ones.
The choice of cache comes down to the question of what exactly you are caching.
If the task is to offload the database on reads: ready-made page fragments, results of heavy queries, responses from external services — Memcached is simpler, cheaper to maintain, and provides more even response time under multithreaded load.
If the cache must survive a restart, store data structures, or participate in coordination between services, the comparison of solutions ends in favor of Redis: it covers these scenarios without additional components.
In practice, both options often coexist: Memcached for the fast read layer, Redis for state and queues.
Migration between them does not require rewriting the application if access to the cache is hidden behind a single interface — then changing the solution comes down to replacing the implementation rather than making edits across the entire codebase.
Case study: we sped up the application threefold with caching
The client — a service with a personal account and an order feed — came with a complaint about a "sluggish" interface.
Measurements confirmed it: the median API response hovered around 900 ms, and at peak, when managers were mass-opening the list of requests, it reached 2.5 seconds.
Meanwhile, the database was executing up to 1,800 identical queries per minute — almost all of them returned the same reference data and aggregates that changed once every few hours.
Memcached was chosen deliberately: we needed fast access to ready-made results by key, without complex structures and dependency-based invalidation.
Our engineers moved the hottest queries into the cache — the user profile, the category reference, order counters — and configured the time to live so that stale data would not be shown for longer than a couple of minutes.
Separately, we designed key invalidation when a record changes: updating an order clears the related aggregates, not the entire cache.
Metrics before and after
| Metric | Before | After |
|---|---|---|
| Median API response | ~900 ms | ~300 ms |
| Response at peak load | up to 2.5 s | up to 700 ms |
| Database queries per minute | up to 1,800 | ~400 |
| Database CPU load | 70–80% | 25–30% |
The speed increased threefold, and the database stopped being a bottleneck: the freed-up load headroom made it possible to launch two new integrations without expanding the servers.
An important side effect — behavior under peak became predictable, and the interface no longer "freezes" during the reporting hour.
A cache is not a "set it and forget it" thing: keys, time to live, and invalidation rules need to be designed for specific data, otherwise the user will see stale information. We document this logic so that the client's team can change it on their own.
Common technical concerns about caching
A cache is always a trade-off between speed and data freshness.
Below is a breakdown of the three concerns that most often come up during the design phase: how data becomes stale, how to maintain consistency between the cache and the storage, and what happens when the caching node itself fails.
How does data in the cache become stale and how is this controlled?
Staleness is not a defect but a manageable parameter. We set the record's time to live (TTL) according to the nature of the data: reference lists and catalogs live long, personal blocks — seconds to minutes.
In parallel, we set up event-driven invalidation: when an entity changes, the application resets its key without waiting for the TTL to expire. The combination of "TTL + explicit reset" eliminates both stuck stale data and unnecessary calls to the storage.
How do you ensure consistency between the cache and the database?
Consistency is built on the order of operations: first a write to the storage, then invalidation of the key — not the other way around, otherwise a stale value manages to get into the cache between the steps.
For hot keys, we apply rebuild locking so that dozens of parallel requests don't hit the database simultaneously, but one rebuilds the value while the others wait for the result.
Where data is critical to freshness, the cache is simply not used — this is a deliberate decision, not an oversight.
What happens if the caching node goes down?
The cache is designed as an accelerator, not as a source of truth: if the node is unavailable, the application keeps working, the load simply shifts to the main storage.
To keep this transition from turning into an outage, we limit the cache response wait time, set reasonable limits on concurrent connections, and test behavior under load before release.
A cluster of several nodes removes the single point of failure, and warming up after a restart brings hot keys back without a spike on the database.
These three concerns are resolved at the configuration stage, not after an incident. Describe your task — we'll come back with an estimate and a plan that will fix the expiration policy, the invalidation scheme, and the behavior on failure.
Let's discuss your caching task
Caching with Memcached solves a specific class of problems: the database stops being a bottleneck on repeated queries. From there, everything depends on what exactly is hurting you — slow catalog queries, degradation under peak load, growing response time after a release.
To provide an estimate and a work plan, we need to understand your picture: the stack, the load profile, the current metrics.
Describe the task — we'll come back with an estimate and a plan. What's worth including in the description:
- where it hurts right now: which pages or methods respond slowly, at what load degradation occurs;
- the application stack and versions of the main components — this determines the choice of client and the mode of working with Memcached;
- how data storage is structured and whether there is already a cache layer (file-based, in-memory, external);
- the load profile: peak requests per second, the read-to-write ratio, the share of authenticated users;
- data freshness requirements — where a stale response for a few seconds is acceptable and where it is not;
- infrastructure: how many application servers, whether there is a dedicated node for the cache, how deployment is structured;
- constraints: data regulations, fault-tolerance requirements, change windows.
Based on these inputs, we determine the key schema, the eviction policy and record time-to-live, the invalidation points when data is updated, and the method for warming up the cache after a restart.
Separately, we assess what happens if the cache node goes down: the application must survive this without a cascading failure, so the plan includes degradation to direct database queries and limiting concurrent requests.
As a result, you receive a stage-by-stage estimate, a work plan with the implementation order, and a list of metrics by which we will verify the result — cache hit ratio, response time on hot pages, database load.
Describe the task in whatever form is convenient: a team lead or engineer on your side plus access to metrics and a test environment will speed up the estimate.