Development of an image object detection system (Object Detection)

Why object detection with YOLOv8 breaks in production

The detector confidently passes validation, but after a month of operation, image detection starts missing objects it previously found. The issue is usually not the weights: the input itself changes — lighting, angle, a new camera, dirty optics, a different field of view.

At the same time, metrics on the holdout set do not move; the discrepancy surfaces only on the live stream.

A change in shooting conditions shifts the feature statistics, and rare classes suffer first: there are only a few of them in the annotations, so a small shift drives confidence below the operating threshold.

YOLOv8 maintains quality within the distribution it was trained on, while frequent categories in the frame dominate — the averaged metric remains attractive while the operator can no longer see missed objects.

Data drift adds its own effect: the composition of objects in the stream changes, while annotations are rarely updated, so detector degradation manifests only after launch.

Weak validation hides this until the very end — a single overall score does not show where detection quality has dropped: on night frames, on small objects, in the far background.

The cost of an error in object detection is a missed defect on the line, an unrecognized license plate, a lost incident in the stream. The model formally works, but the process no longer relies on it.

We break down how this is addressed: separate metrics by class and subgroup, input drift monitoring, and a retraining policy.

Implementation effect: processing speed, accuracy, robustness

Before implementing the detector, the image array was reviewed selectively: the operator viewed part of the frames, and the rest went to the archive unchecked.

With a stream of several thousand frames per day, selective control produced a miss rate of around 15–20% — the object was found at a stage when fixing it was already expensive, and the cause was sought manually through logs.

After launching the YOLOv8 model, review proceeds as a continuous stream. The archive accumulated over a month is processed overnight instead of weeks of manual work; new frames enter the queue with a delay of seconds rather than hours. Image processing speed stops being the bottleneck — only the upload channel capacity becomes one.

Detection accuracy grows not “by itself,” but from calibrating the confidence threshold on the customer’s data. We annotate ambiguous frames, retrain the model on real shooting conditions, and set thresholds separately for each object class.

The share of false positives drops severalfold: the operator stops closing identical false alarms, and the freed attention goes to real cases.

We test detector robustness on shifts that break laboratory metrics: lighting changes, season, a new camera angle, partial occlusion of an object.

The failure class “model is silent on an unfamiliar frame” turns into the failure “model is uncertain” — such frames go to the manual review queue instead of being lost.

The share of manual inspections and the share of misses are visible on the monitoring dashboard, so we catch degradation before it becomes a problem.

We work on an open stack around Ultralytics: the customer is not tied to a single vendor and retains control over the model, annotations, and thresholds.

Detector operation modes: batch, streaming, on-device

The operation mode is determined not by the model, but by where frames come from and what happens to the result.

The same current YOLO branch in batch, streaming, and local execution produces different latency, different load on the compute unit, and different requirements for the communication channel. That is why we first fix the scenario, then choose the mode, and only then configure the model.

In batch mode, the main lever is frame grouping and a single resolution: throughput grows severalfold, and processing order is set by queue priority.

In streaming mode, one configures the frame skip step, confidence threshold, and object tracking between frames; otherwise, a single object is counted multiple times.

On-device detection solves operation without a network: events are written locally and go to the server when a channel appears — no frame loss occurs during a communication outage.

Modes can be combined: the device filters the stream, the server rechecks ambiguous frames, and the accumulated annotations go into a batch retraining dataset.

We go through your scenario and say which mode will withstand the load and which will hit the channel or device memory limit.

Work stages: from data audit to model handover

Detector quality is determined not by the chosen architecture, but by the data and the order of work. We break the project into stages so that each next one relies on a verified result of the previous one: an annotation error discovered after training costs weeks of rework.

1. Dataset audit. We look at what already exists: classes and their balance, duplicates, corrupted frames, spread of shooting conditions — light, angle, background. The output is a report on which scenarios are missing and how many frames need to be collected.
2. Solution design and acceptance criteria. We fix what we consider an error — a missed object or a false positive — and agree on which metric defines the result. This protects against disputes at the final stage.
3. Image annotation. A policy for specific classes, cross-checking of ambiguous frames, control over the share of missed objects in the training part.
4. Training and retraining. We start with a pretrained YOLOv8 model, select augmentations for real shooting conditions, then retrain on added data until reaching the target metric.
5. Acceptance on a holdout set. We test on frames that did not participate in training: another shift, camera, time of day. This shows robustness rather than memorization of the training set.
6. Model handover. We provide weights, inference configuration, trigger thresholds, run scripts, and documentation for retraining on new data.

After handover, your team can retrain the model independently — the policy and scripts are included. When the data volume grows, we will return with an expansion estimate.

Deliverables: configurations, schemes, policies

The delivery package is not a folder with model weights, but a set of artifacts by which your team can launch the system on its own, update it for new shooting conditions, and hand over maintenance to another engineer without us. Below is what exactly is fixed in the repository and documentation by the time of handover.

• Model configuration — settings files with input size, confidence and non-maximum suppression thresholds, class list, and binding to a specific weight branch: the same run produces a reproducible result on the test set.
• Integration scheme — data flow from the camera or storage to the model and back, with failure points, frame buffering, exchange formats (image, bounding box coordinates, labels, confidence), and behavior when the source is unavailable.
• Retraining policy — frequency, requirements for new annotations, acceptance criteria for a version, and conditions for rollback to the previous one if per-class accuracy drops.
• System documentation — architecture description, deployment diagram, instructions for launch, diagnostics, and typical failures, including analysis of false positives.
• Data access policies — roles, differentiation of rights to source frames and annotations, retention periods, depersonalization rules before export to an external perimeter.
• Quality control scripts — calculation of precision and recall by class, a set of reference frames, a run log: after any change, it is visible whether things got better or worse.
• Pinned environment — dependencies and build containers with exact versions, so deployment on your perimeter does not turn into selecting libraries.

Handover includes acceptance: we run the reference set together with your engineer, show metrics for each class, and answer any item on the list — the system should not remain a black box dependent on the contractor.

Case study: reduced missed objects and archive review time

Before

The operator manually reviewed the archive for a shift and marked the required events in the log. On a dense stream, this produced 18–22% misses, and on small and partially occluded objects the error rate grew: the eye gets tired, and context between frames is lost.

A full review of a shift took 6–8 hours — the recording became outdated before it could be reviewed.

What we changed

We annotated the stream by class and retrained a detector based on YOLOv8 on real frames from the site, including night and blurred ones.

On top of the model, we added post-processing: linking frames into chains, separate confidence thresholds for each class, and geometric filtering of false boxes. The run was made batch-based — a queue of frames is processed rather than one frame at a time.

Results

The miss rate fell to 3–4%, false positives decreased by about three times, and the operator checks selected windows instead of the entire array. Reviewing a shift takes 40–50 minutes instead of 6–8 hours.

The confidence threshold is configured separately for each class, so sensitivity rises without an increase in false positives — in practice this yielded more than the accuracy gain of the model itself.

How YOLOv8 differs from other detector families

Detector families differ in how they search for an object in a frame, and their strengths depend on this. One-stage models — YOLOv8, SSD, RetinaNet — output boxes and classes in a single pass, so they handle a camera stream without accumulating latency.

Two-stage schemes (Faster R-CNN) first propose candidate regions and then refine them: they win on small and overlapping objects, but process a frame noticeably longer.

A separate branch is transformer-based detectors (DETR and successors). They are more robust in dense scenes where objects stand close together and depend less on manual post-processing tuning, but they require a larger volume of annotated data and longer training.

YOLOv8 provides an advantage where stream speed is critical: video surveillance, product display control, vehicle counting, operation on an edge device without a server.

The model retrains quickly on a small set of annotated frames, behaves predictably in terms of latency, and covers most tasks where the object occupies a noticeable part of the frame.

It is wiser to take another family when the priority is quality on complex data: micro-objects, defects in large images, heavy overlaps, high cost of a miss. Here a two-stage or transformer-based setup often outperforms YOLOv8 in recall, while speed moves to the background.

There is also a third path — instead of training a new model, retrain an already deployed one if its classes are close to your task. We compare options on your frames: per-class recall and precision, latency on the target hardware, and the volume of annotations required. We select the family for the task, not out of habit.

Technical concerns: annotation, retraining, operation in an isolated environment

Annotation, retraining, and operation without an external network are questions that come up before the start, not after. Below are brief answers to them, using object detection projects with the YOLOv8 model as an example.

How does the model behave on night, overexposed, and blurred frames?

As it was taught: if the sample lacks night, rain, glare, and small objects, errors will appear exactly on them. That is why we assemble test sets by shooting conditions and calculate metrics separately for each subgroup — the failure area is visible, not just the average number.

During training, we feed distortions and noise so the model does not fall apart on a non-standard frame.

Can the system be deployed in an isolated environment?

Yes. Training and inference run on your hardware: weights, dependencies, and updates are delivered through an internal registry; neither a frame nor a label leaves the perimeter.

For a fully isolated environment, we prepare a portable kit for manual installation and test it on your bench.

What happens with retraining on your own data after launch?

Frames where the model made a mistake or triggered with low confidence go into the annotation queue, then into retraining. We number weight versions; rollback to the previous one is one command.

We hand over the mechanics to your team along with instructions, so the update cycle does not depend on us.

Let's discuss the task: what to send for an estimate and plan

For the detection project estimate to rely on facts and the work plan to rely on real conditions, the entire data archive is not needed. A sample and a description of constraints are enough. Below is what to send to discuss the task.

• a sample of 50–200 files — images or short videos, including difficult frames: poor lighting, occlusions, small objects;
• a list of target classes and annotation rules: what we consider a detection and whether a box around the object is needed;
• shooting conditions: camera angles, distance, motion — this determines whether a current-branch YOLO family model is enough or retraining on your data will be required;
• where model inference is computed: a server with an accelerator, cloud, or an on-site device, and what frame processing latency is acceptable;
• the integration point: where the event goes — to a web service, message queue, or an existing system — and in what form;
• quality requirements: the acceptable share of misses and false positives and your annotated set for acceptance testing;
• data constraints: whether frames can be taken outside the perimeter and whether operation entirely on your hardware is required.

With this set, we come back with a timeline estimate and a stage-by-stage work plan: data and annotation review, baseline run, retraining for your classes, integration, and acceptance on the agreed metrics.

If some data is not ready yet, we will start with what is available so that the estimate is still substantive.