Why video analysis of sports matches fails
The first failure is visible in overlaps: in a duel or a dense group near the goal, two players merge into one track, the identifier jumps to a neighbor, and player detection accuracy drops exactly where the match is decided.
Then the track breaks into short fragments — and distance and acceleration statistics are calculated from scraps, not from the episode.
The second problem is field marking. Homography by lines holds while the camera is stationary: when panning and changing angle, the lines go out of frame, the marking shifts, and player coordinates shift along with it. Heatmaps and activity zones after that show not the game, but accumulated calibration error.
Lost episodes finish the picture. The system does not record a substitution, a disputed ball, an off-camera foul — and the coach still sits down for manual review of the recording: scrolls half by half, marks up manually. One match turns into hours of routine, and the review result is not repeatable from shift to shift.
All this runs into a processing bottleneck. Several synchronized cameras produce a stream of frames that must be detected, matched between angles, and have identifiers maintained — the queue grows faster than it can be processed.
A solution built "head-on" delivers the report by morning, when the review is no longer relevant.
Accurate systems like Hawk-Eye show that the task is solvable, but require precise calibration and computational headroom. We explain how this is addressed.
What changes after implementing a review system
The first matches after activation show the main thing: review stops being manual. Tracking completeness on overview cameras exceeds 95% — the system holds a player through intersections, turns, and short overlaps where an operator lost him for a few seconds.
It is this class of failures — loss of binding a number to a player — that goes away first, and reports no longer have "white spots."
Already in the first week, match review speed changes: cutting game segments, binding events to the timeline, and finding an episode by player or zone take minutes instead of hours of manual viewing.
The coach formulates a query in words — "all attacks down the right flank in the second period" — and gets a selection instead of reviewing the entire recording.
By the first round of matches, enough volume accumulates to enable automatic reports: distance and number of accelerations per player, activity zones, time in lines, duels.
Frame analysis is built on the open-source computer vision library OpenCV, so metrics are calculated the same way for all matches — comparison between matchdays and opponents becomes valid.
Player heatmaps stop being a one-off exercise: they are collected automatically for every match and fit into a unified format.
This makes it possible to overlay maps from different matchdays and see how a footballer's area of action changes after a formation change, rather than arguing about it by eye.
Then the process itself changes: manual correction is reduced to isolated cases — halftime substitutions, disputed episodes, refereeing decisions.
Binding errors that previously stretched through the entire match remain local and are fixed in seconds, so the staff gets the review immediately after the game, not by the next training session.
Processing modes: streaming, batch, on-premises, and cloud
The processing mode determines what the team gets from the recording and how quickly.
Streaming processing delivers events during the match — counters, zones, prompts to the coach at halftime; batch review works on already recorded material and provides depth: heatmaps, distance, episode analysis.
The bottleneck here is usually not the model, but frame delivery: if the channel degrades, it is not the metrics that are lost, but the object track itself, and it cannot be restored from recording fragments.
That is why the mode is chosen not by processing "power," but by the scenario and site constraints — who is allowed to export the recording, what the channel to the object is, and whether the result is needed in the moment.
| Option | When it fits | Limitations |
|---|---|---|
| Streaming processing | live events, scoreboard, prompts to the coach during the match | requires a stable channel; if it degrades, frames are lost and with them — track continuity |
| Batch review | post-match review, heatmaps, reports for the entire match | result is not in the moment: between the final whistle and the report, time passes for upload and calculation |
| On-premises processing | closed facilities, restrictions on exporting recordings, stable stream from several cameras | requires own computing capacity and its maintenance; sustaining peak tournament-day load requires more resources |
| Cloud processing | season with uneven load, one-off tournaments, distributed venues | sensitive to channel quality on upload: compression loses fine detail — jersey number, field markings |
| Hybrid mode | events and tracking — on-site, heavy analysis and storage — off-site | more complex to set up: need to agree in advance which data and in what form leave the perimeter |
In practice, most sports match video analysis projects converge on hybrid mode: it removes the main risk — dependence of the result on the channel at game time. At the start, we fix what is processed on-site, what goes off-site, and how we check frame loss at each section.
Implementation stages: audit, design, setup, verification, handover
The order of work here is not bureaucracy, but a way not to spend months on a model that has nothing to process.
First, engineers analyze what is actually recorded by your cameras, and only then design the scheme — this way the pipeline is not built for an abstract "ideal" match.
-
Recording audit. We analyze the source material: angles, frame stability, overlaps, lighting during the day and under floodlights. The result is a report: where the task can be solved with existing cameras, and where an additional angle is needed.
-
Processing design. We determine what is calculated in the stream and what after the match from the recording, where coordinates and events are stored. We fix exchange formats so that recognition, player tracking, and statistics can change independently.
-
Camera calibration. Binding to the field coordinate grid, distortion compensation, stream synchronization by timestamps. Without this, the heatmap drifts, and the positions of one player from two cameras cannot be brought to a single point.
-
Control episodes. Together we select 20–30 fragments: counterattacks, dense groups near the goal, disputed moments. On them we fix the metric and acceptance threshold before development starts.
-
Pipeline setup. We fine-tune models on your material, configure trigger thresholds and event logic: pass, shot, zone entry.
-
Verification. Run control episodes, compare with reference markup, report on the share of correct detections and error classes.
-
Project handover. Documentation on the scheme, configuration files, regulations for fine-tuning on new matches, and training for your team.
Markup schemes, detection models, and video analysis regulations
After analytics launch, the customer retains not only a working service but also a full set of engineering artifacts. Without them, the system turns into a black box: it is unclear by what rules metrics are calculated and what to change when shooting conditions change. Below is what exactly we hand over and go through with the team.
- Processing scheme. Camera streams, order of stages — decoding, detection, tracking, aggregation — failure points, and storage locations for intermediate data.
- Detection models. Configurations with confidence thresholds, class lists, and angle constraints; comments on parameters so fine-tuning does not start from scratch.
- Frame markup. Markup rules and a reference set of frames with a replenishment procedure — a new model is compared on the same set, not on "similar" matches.
- Metric calculation. How heatmaps are built: averaging window, normalization by possession time, outlier filters, export formats for the coaching staff.
- Access policies. Roles — coach, analyst, administrator: what each sees, recording retention periods, video access log.
- Operator instructions. Starting match processing and actions for dropped frames, lighting changes, and track desynchronization.
- Verification regulations. Control matches, comparison of calculations with manual markup, permissible discrepancy share, and procedure for rolling the model back to the previous version.
We go through each item on the customer's match, not on a demo. After that, the team handles fine-tuning and regulation edits independently.
Case study: tracking completeness and review stability
Before
A match lasts an hour and a half, while the analyst's review stretched to a day. Detection lost players in group duels and during quick reorganizations, the ball disappeared on long passes, and tracks merged on intersections.
Tracking completeness on the test set stayed around 71%; over forty lost episodes accumulated per game — key phases were searched manually from the recording.
What was changed and what was achieved
The pipeline was rebuilt: homography calibration by field markings, multi-scale detection, matching tracks between frames taking kinematics into account, a separate model for the ball.
Frame processing primitives are built on OpenCV, model execution runs on a node next to the venue — the review is partially ready by the final whistle.
Tracking completeness rose to 96%, lost episodes dropped to 3–5 per match, and almost all are in disputed moments with camera overlap.
Review preparation time decreased from 9 hours to 2, manual operations dropped by about 80%: the analyst verifies automatic marks instead of marking up from scratch.
Metrics were calculated on reference markup: two matches were marked up manually, compared with the model output frame by frame, and errors were broken down into misses, false positives, and track merges.
After fixes, we tested on recordings from another team and another venue — so quality would not depend on a single stadium.
How does detection differ from tracking in game analysis?
Detection answers the question "what and where is visible in the frame right now": the model finds players, the ball, goals, and markings, and outputs bounding box coordinates with classes.
Tracking adds time to this — it links boxes between frames into trajectories where the object has a persistent number. Without such linking, two players in a duel swap places, and all subsequent statistics are wrong.
On top of trajectories, player identification works: a track is bound to a specific person by jersey number and team roster.
This is where typical failures live — the number is covered by a hand, the player goes into an overlap, the camera angle changes; after that the track needs to be rebound, not started from scratch.
Additionally, positions are transferred to field coordinates by calibration from marking lines; otherwise there is no basis for calculating meters and zones.
Heatmaps are not a separate model, but an aggregate of trajectories: positions accumulate by zone, and from them time in zone, distance covered, and number of accelerations are calculated.
With broken calibration, the image comes out smooth, but coordinates do not correspond to the field, and the review is harmful.
| Layer | What it solves | What breaks without it |
|---|---|---|
| Object detection | Finds players, ball, goals in the frame | Misses and false boxes go into the report as is |
| Player tracking | Links boxes over time into trajectories | Players swap places, distance is calculated manually |
| Player identification | Binds a track to number and roster | Personal metrics get mixed up between players |
| Heatmaps | Brings positions into field coordinates | A shift is visible that did not exist |
This is assembled bottom-up: stable detection, then tracking, then identification, and only then aggregates. An error in a lower layer cannot be fixed by an upper one, so accuracy is checked at each layer separately and compared with manual markup of a match fragment.
Common doubts: frames, markup, and data privacy
Engineering doubts about match video analysis usually come down to four: is one camera enough, what to do with overlaps in the fight for the ball, how to verify tracking accuracy before launch, and can frames be processed without taking them outside the perimeter.
Below are direct answers, including cases where everything honestly comes down to the quality of the source footage.
Is one camera enough?
It depends on the task. For movement heatmaps and calculating time in zones, one wide-angle camera and a stable point in the stands are enough.
Analyzing duels and tracking the ball in a dense group require a second viewpoint: overlaps create track breaks, and the episode can no longer be restored from a single angle.
With two cameras, we calibrate them using the common field markings and stitch tracks by time — otherwise player coordinates diverge.
What to do about player overlaps?
Detection tracks people by identifiers, and in contact the track continues by trajectory prediction and re-identification by external features — jersey, number, equipment color.
Episodes with low confidence are flagged: the operator sees where the model is unsure and confirms manually. This is less labor-intensive than marking up the entire match again.
How to verify accuracy before launch?
We run the models on a held-out sample of recordings from your venue and compare with manual markup of control segments. The output is the share of correctly maintained tracks and areas where the system consistently errs: shadows, backlight, motion blur. These numbers show whether a second camera or scene improvement is needed.
Do frames leave the club's perimeter?
No. The on-premises system is deployed on your equipment, video is processed locally and is not transmitted outside — neither the stream nor archives. Only aggregates go outside: coordinates, events, metrics.
Access to storage is granted by roles, and we keep an access log, so data privacy remains under your control.
Let's discuss the task: what is needed to start work
An accurate estimate comes not from a task description, but from the material itself: recordings show how many cameras are in view, how the light behaves, whether there are player overlaps, and whether the field is marked up.
Therefore, to start, an archive of two or three matches in different conditions and a list of metrics needed by the coaching staff or analytics department are enough.
- match recordings — 2–3 games, including difficult ones: evening light, rain, dense duels in the penalty area;
- list of metrics — player and ball tracking, movement heatmaps, event detection: passes, shots, possession;
- processing mode — real-time review from camera or from archive, permissible delay in delivering the result;
- venue constraints — camera installation points, height, viewing sector, access to wired network and servers;
- data and access — examples of existing markup, competition regulations, requirements for recording storage and privacy;
- result consumer — who views reports and in what form: operator panel, table export, integration with the club's accounting system;
- target indicators — what recognition accuracy you consider sufficient for decisions to be made based on this data.
Reviewing the submitted material is the task statement itself: we look at which scenes cause failures and separate what is solved by model tuning from what requires additional cameras.
Then we fix the scope of work by stages — data preparation and markup, training and calibration, metric output, test run on a closed match.
We return with an implementation plan and estimate timelines within a few business days after receiving the recordings; on a call we go through the risks and acceptance procedure.
