Why a recommender system stops hitting user interests
A recommender system learns from impression logs, and those logs are the result of its own previous decisions. A closed loop emerges: popular content is shown more often, collects more interactions, becomes more entrenched in ranking weights, and receives even more slots.
After several retraining cycles, the quality of the feed begins to reflect not the audience's interests but what has already been shown.
This is how popularity bias forms. The long tail of the catalog falls out of rotation: no impressions — no signals — the model cannot estimate relevance and does not move the item up in the feed.
New items fall into the same dead end: their features have not yet accumulated statistics, so the algorithm plays it safe and gives the slot to proven content.
Then catalog cooling starts: the effective feed contracts to a narrow core, and the rest stops bringing views.
Externally, this shows up as falling completion rates — the next recommendation resembles the previous one, the user leaves earlier, the share of sessions completed to the end declines, and churn after the very first session grows.
Niche audiences suffer most: their interests do not find their items and do not return.
Tweaking a single coefficient in ranking does not cure this — you need to break down the signal sources, the impression policy for new items, and how the system explores the unknown alongside exploiting the proven. Let's look at how this is closed.
Implementation effect: completion rates, churn, feed response time
After rebuilding the feed, what changes is not the "feel of recommendations" but concrete numbers.
Watch depth grows because the selection is assembled based on the current session rather than daily popularity statistics: the viewer is shown a continuation of what they are watching now, not what everyone watched yesterday. Audience retention grows most in niche catalogs, where overall popularity poorly predicts interest.
The share of returning viewers is the second measurable effect. A whole class of failures disappears: the cold start of a new profile is covered by content features and device context, and rare content stops falling out of the selection because of low view counts.
Previously, these two cases produced an empty or nearly empty feed — now the viewer gets a meaningful list on the first visit.
We keep feed response time low by separating precomputation and online reranking: heavy features and vector representations are computed in advance, and nearest-neighbor search on Faiss returns candidates in fractions of a second.
Selection preparation is reduced from hours to minutes, so new models and rules take effect the same day rather than in an overnight recomputation.
We measure the effect on control groups: watch depth, return rate, eliminated feed failures, and response time at a high percentile, not just on average. This shows that the speedup is not bought by degrading selection quality.
How a recommender system is built: three operating modes
The deployment scheme determines three things: how quickly the model updates, where viewing history and profiles physically reside, and who is responsible for response latency at feed time.
In practice, you choose between an in-house environment, external infrastructure, and a hybrid — the difference shows up not in recommendation quality but in training speed and in the constraints you accept along with the option.
| Option | When it fits | Constraints |
|---|---|---|
| In-house environment | Behavioral data must not leave your infrastructure, load is predictable, peaks are known in advance | Retraining is limited by hardware capacity; external signals and part of the context are unavailable, and the model updates less often |
| External infrastructure | You need a fast start and frequent experiments with training on large volumes of history | Viewing history leaves your perimeter; response depends on the channel to the external platform |
| Hybrid setup | Sensitive data stays inside, while heavy training and infrequent recomputations are moved outside | More expensive to maintain: two environments, feature synchronization, model versioning, and discrepancy control between them |
In all three cases, model training follows the same scheme: history collection, feature preparation, scheduled or triggered retraining, validation on historical logs, and testing on part of the traffic.
Only the location of the heavy stage and the time between the appearance of new data and the feed update change.
Implementation stages: from data audit to model handover
A video recommender system is not a single model but a bundle of feature preparation, the serving service, and display rules. That is why the work proceeds in stages: each next one relies on a verifiable artifact from the previous one, not on assumptions.
- Data audit. We examine which events are captured from the player, with what completeness and delay, and how the catalog is labeled. We check logs for gaps and duplicates: a model trained on dirty events reproduces their errors.
- Flow diagram. We fix the entire event path: collection, storage, feature computation, training, feed serving. We agree on where features are computed in batch and where they are needed on the fly.
- Assembly and training. We prepare the ranking model and the scaffolding around it: time-based splitting instead of random, deferred validation, and a separate set for rare scenarios.
- Feed tuning. A layer on top of the model: diversity, freshness, repetition limits, cold start for new videos, and rules for licensed and children's content.
- Live traffic validation. First a shadow mode — the system computes the feed but does not affect it. Then comparison with the current feed on part of the audience: watch time, completion rate, drop-off in the first seconds.
- Model handover. We hand over code, configurations, the retraining procedure, and metric dashboards, and go through typical failures and the procedure for rolling back to the previous version with your team.
Each stage ends with something verifiable: a data report, a diagram, a working feed on part of the traffic. That is why handover happens without surprises, and your team sees from day one which metrics the system lives by.
Handover artifacts: flow diagrams, configurations, procedures
Development ends not with a demo but with a kit that your team operates and develops without us. Everything we build is handed over as artifacts tied to your environment: launch parameters, data movement diagrams, feature descriptions, and the rules by which the system updates.
We validate the kit on the live environment — we run typical scenarios together with your engineers so that handover does not remain on paper.
- Configurations for serving and retraining services — parameter files for ranking, source weights, candidate sample sizes, and cutoff thresholds. They change without rebuilding, with an edit history and a clear rollback point.
- Data flow diagrams — from viewing events and reactions to candidate generation and ranking. They indicate where data is buffered, where it is aggregated, and what happens if a node is lost.
- Feature dictionary — a catalog describing each characteristic: source, type, aggregation window, behavior on missing values. The same feature is computed identically in training and in production.
- Access policies — roles for services and people, visibility boundaries for user data, environment separation. It separately describes which data does not leave your environment.
- Update procedure — retraining frequency, conditions for automatic promotion of a new version, the set of quality checks before rollout, and rules for returning to the previous one.
- Monitoring and response procedure — control over the share of successful feeds, feature freshness, and selection quality, with trigger thresholds and an incident triage procedure.
- Operations documentation — how to bring the service up from scratch, how to add a new candidate source, and how to reproduce training on historical data.
After acceptance, the artifacts live in your repository, not in a chat. Let's discuss the task — we will show what such a kit looks like on a specific example of a streaming catalog.
Case study: how a streaming service reduced churn and increased completion rates
The first version of the service's feed was built on global daily popularity plus manual editorial selections. A new title did not rise to the top positions until it gained views on other platforms, and a viewer with one session per week saw the same row as an active subscriber.
Churn concentrated in the first weeks: a person opened the catalog, did not find "their" content, and did not return.
What was changed in data and rules
First, we assembled an event layer: completion accurate to the minute, pauses, rewinds, drop-offs in the first minutes, additions to the list.
Events were stored in ClickHouse so that the viewer profile could be recalculated immediately after a session rather than once a day — the feed had time to react the same evening.
On top of the rebuilt ranking, we added rules: a quota for new titles, a ban on repeating one topic in a row, demotion for content the viewer had already abandoned, and a separate cold-start branch by interface language and device type. This removed the bias where four or five of the most popular titles lived in the feed.
Metrics before and after
- Completion of the first episode: 41% → 63%
- Share of viewers who returned within a month: 28% → 44%
- Share of new titles in the first ten positions: 7% → 31%
- Repeated impressions of one title per session: 4.1 → 1.3
We did not touch the player or the cards — the shift came specifically from rebuilding the feed.
The result was recorded at week eight, and a control measurement a quarter later showed that the metrics held: the rules do not have to be tweaked manually, and the share of manual selections in the feed fell from 60% to 15%.
How does collaborative filtering differ from content-based?
Collaborative filtering relies only on the interaction matrix: who completed what, liked it, added it to a list, left at which minute. It does not need descriptions — it finds connections that labeling cannot express.
The price for this is cold start: a new film and a new viewer without history get popular content instead of relevant content.
The content-based approach works with features of the object itself: genre, tags, actors, director, duration, a vector representation of the description and subtitles.
It turns on from day one and is explainable, but it depends on labeling quality, and the viewer profile closes in on what has already been watched — the feed stops surprising.
In a working feed, they are combined.
Collaborative filtering provides the main candidate partition, the content-based approach covers cold start and niche genres, and behavioral signals from the current session — completion of the first minutes, skip, autoplay to the next episode — outweigh months of history. Netflix describes its feed as a mixture of models where the weights depend on the viewer segment.
We validate the comparison not by eye but by metrics: share of completions to the end, how many new catalog items the viewer opens per month, subscription retention after a week and a month, and feed response time. These slices show what share each model should occupy in the combined feed for a specific segment.
Common technical concerns: cold start and data quality
Before starting, almost everyone asks the same things: what to do if there is no user data, how not to break the feed during an update, and whether the models will remain with the client after launch. Briefly — how this is arranged; we will go through the details on a call.
If there is almost no user history, will recommendations be random?
No. We cover cold start with content features (topic, duration, genre, author) and popularity in a similar segment, and we pull in the first actions within minutes. Personalization turns on gradually, without "empty" selections at entry.
What to do about noise in labeling?
We assess labeling quality before training: we clean duplicates, bots, and accidental clicks, and we weight signals by watch depth rather than by the mere fact of opening. This removes the bias toward "clickbait" cards.
Can the model be updated without stopping the service?
Yes. The new version is rolled out in shadow mode on part of the traffic and compared with the current one by retention metrics; if it degrades, rollback is instant. Updating without stopping is the standard mode, not a separate operation.
Who owns the models and data?
The client. We hand over model artifacts, configurations, feature schemas, and retraining instructions — the team can develop the system itself. Model ownership is not tied to us.
Why do selections look samey?
That is how optimization for clicks alone works: the feed collapses into a narrow circle of popular content. We add a repetition penalty and diversity by topic and author — content coverage grows, and retention does not fall.
Let's discuss the task and come back with an estimate and implementation plan
So that the work estimate resembles reality, on the call we break down the task in terms of data and metrics, not wishes.
Describe which signals are already collected — impressions, clicks, completions, search queries, ratings — and we will say which class of models is realistic on them, and where history will have to be collected from scratch. This also determines the scope of work and the time to the first measurable effect.
- which signals exist and over what period history is retained — this determines how the feed will behave on new catalog items;
- what we consider success: completion rate, session depth, next-day return, or feed diversity;
- where exactly recommendations are embedded: home page, "similar" block, autoplay, selections, emails;
- how many positions need to be ranked at once and how quickly the catalog updates — this determines the architecture;
- what infrastructure and data rights constraints apply on your side today;
- who on the client side makes decisions about feed quality and how often measurements are planned;
- which scenarios must not be broken: content rights, age restrictions, regional limits.
After the call, we send an implementation plan by stages, a work estimate for each part, and a scope of work with responsibility zones: what we do and what remains with your team.
The first step is a pilot launch on one catalog slice, where the effect is visible in a couple of weeks; the scope of the main implementation is fixed after it. Describe the task in a convenient form — we will come back with an estimate and plan.
