Traffic Sign Recognition: Where It Breaks on Real Roads
On labeled datasets, traffic sign recognition systems show near-perfect accuracy, but on real roads they start making errors.
The gap arises not from the model but from the difference between the test set and the camera stream: lighting changes, exposure, road surface condition, and the signs themselves. We break down exactly where sign detection fails on video.
The first group of failures is light. Low sun in backlight creates glare on the sign, and the sign turns into a white spot without a readable outline; at night, glare from headlights and overexposure from oncoming sources add to this.
In winter, the picture worsens: a snow-covered sign loses its shape, and a sign tilted after wind leaves the zone of confident detection.
Difficult weather conditions work together with road marking quality. Worn, snow-covered, or traffic-blocked markings disrupt association: the sign is found, but it is unclear which lane it belongs to, and there is nothing to base a maneuver decision on. The same uncertainty occurs when a sign is blocked by a truck or disappears beyond a bend.
A separate topic is false positives. Advertising structures with similar geometry, bright banners, stickers on vehicle bodies trigger events where there is no sign at all.
While filtering by shape, color, and position in frame is weak, the operator receives a stream of questionable alerts and stops trusting the system.
Next we break down how each of these failure classes is addressed — at the level of frame capture, preprocessing, and post-filtering of events.
Measurable Effect: Accuracy, Robustness, Response Time
Detection accuracy and recognition recall are measured not "by eye" but on a held-out set of frames labeled according to GOST R 52290 — separately for signs, separately for markings and stop lines.
Metrics are split by class: rare priority signs and temporary plates fail first, so reporting for them is separate. The outcome of implementation is the share of frames where the system confidently outputs an object, not an average figure across the entire set.
Weather robustness is tested with scenarios, not average accuracy: oncoming sun, twilight, rain with streaks on the windshield, snow, a sign caked with dirt, a shadow from a semi-truck.
For each scenario, the miss rate and false positive rate are recorded — they differ by an order of magnitude and determine whether a second data source is needed at a specific section.
Frame processing time is a budget divided between object detection, tracking it between frames, and filtering false detections.
If a frame does not fit within it, the system either loses pace or "catches up" and outputs coordinates with a delay — at speed this shifts the reaction point. After pipeline optimization, the variation in time per frame drops, and latency becomes predictable.
Reduction in manual corrections is calculated as the share of frames the operator annotates again themselves.
A closed loop — the system marks uncertain frames, the annotator confirms or corrects, the corpus goes into incremental training — shifts this share severalfold over a few iterations.
Then acceptance proceeds by metrics on a fixed set of scenarios, regression is run on every model update, and disputed frames go into the training corpus rather than manual review.
Recognition Approaches: Comparison of Operating Modes
The choice of mode is a trade-off between reaction latency, channel robustness, and the cost of compute hardware in the vehicle.
If communication at a section drops and a decision is needed within fractions of a second, all logic moves onboard; if the fleet is heterogeneous and the model is updated frequently, it is cheaper to compute on the server. Below is a comparison of modes that we use in traffic sign and marking recognition projects.
| Option | When it fits | Limitations |
|---|---|---|
| Onboard processing | Highways and sections with unstable connectivity, requirements for instant reaction, operation in tunnels | Hardware in the vehicle is limited, model updates require a distribution procedure across the fleet |
| Server processing | Urban fleet with a stable channel, centralized monitoring, frequent model updates | Sensitivity to connection quality and channel latency, volume of traffic from cameras |
| Hybrid scheme | Mixed fleet: onboard — fast candidate selection, on the server — verification and analytics | Harder to debug: model versions onboard and in the environment must be aligned |
A separate axis is regional incremental training: the set of signs, marking condition, and seasonal conditions differ, so we refine the base model on a local frame sample.
In a hybrid scheme this is most convenient: the updated model is distributed onboard without reflashing, and disputed frames go to review and return to the training set.
The mode is selected based on your section and fleet — we discuss this before start so the environment does not have to be reworked later.
How Implementation Proceeds: Stages of Engineering Work
The sequence of work is the same for any site — only the scope changes: a highway, a toll road, and an enterprise territory have different densities of signs and markings. Implementation proceeds in iterations, and after each stage a verifiable artifact remains.
- Road scene audit. We analyze the route: which signs and marking types occur, under what lighting, where a sign is blocked by bushes, a semi-truck, or dirt. The output is a list of classes and scenarios that the system must cover.
- Dataset collection and labeling. We use real recordings from the site, not only open datasets: each region has its own support shapes, duplicate signs, temporary signs at repairs. We label with cross-checking — otherwise the model will learn an error as the norm.
- Model training and validation. We keep part of the data separate and do not show it during training. We look not at overall accuracy but at behavior by each class: an error on a rare sign costs more than on a common one.
- Threshold tuning. We select the confidence at which the system outputs a decision and the confidence at which it passes the frame to the operator. For an assistant, it is more important not to miss; for violation recording, not to produce a false positive.
- Acceptance on a control section. We test on recordings that were neither in training nor in tuning: night, rain, glare from headlights. We count misses and false positives by class and record them as a baseline for comparison after updates.
- Handover to operation. We provide the configuration, deployment scheme, procedure for incremental training on new data, and rollback to the previous model version.
Delivery Package: Configurations, Schemes, and Procedures
The delivery package is not an archive of files but a set of artifacts your engineers can use to run the system without us: deploy it on a new environment, change the activation threshold, add a sign class.
Each item is checked at acceptance — if the document cannot be used to reproduce the configuration or bring recognition up from scratch, the item is considered incomplete.
- System configuration: confidence thresholds by class, list of recognized signs and marking types, camera binding to zones and lanes, duplicate rejection rules. Stored in version control — changes are traceable to author and date.
- Data flow schemes: the path of a frame from capture to event recording, buffering points during channel interruption, queue behavior under overload, composition and format of the output event for adjacent systems.
- Access policies and roles: who changes thresholds, who only views metrics, who exports frames for review, how administrator action logs are maintained.
- Model documentation: class composition, accuracy and recall for each on a held-out set, applicability conditions and degradation boundaries — night, rain, contaminated or partially occluded sign.
- Maintenance procedure: frequency of quality checks on fresh data, procedure for adding a new sign class, conditions for rollback to the previous version and recovery time.
- Error review procedure: incident card with frame and verdict, criteria for including a frame in the training set, responsible person and response time.
- Set of reference frames and acceptance protocol: fixed scenes by which quality is measured before and after any change.
The package answers three questions: how the system is configured, why it works this way, and what to do in case of failure. We hand it over together with training for your engineers — describe the task, and we will come back with an estimate and plan.
Case: Accuracy on a Complex Section Before and After
The project was launched on a section with dense traffic and frequent surface changes: asphalt, concrete slabs, prolonged repairs with worn markings.
We worked with Operator of the Regional Road Monitoring Network — before implementation, the general model recognized signs in 82% of frames during the day, and in rain and backlight it missed up to 14 signs per 100.
False positives per kilometer were about six: the system reacted to advertising signs and signboards similar in shape.
What Drove the Gain
Three changes. First — incremental training on rare classes: temporary signs, signs in covers, worn and water-flooded markings. Second — object confirmation across several consecutive frames so that single flashes of the model did not reach notification.
Third — a separate processing branch for night and precipitation with automatic frame exposure adjustment.
How We Tested
A control section 12 km long was manually labeled by two independent operators, disputed frames were sent for a third review. Metrics were taken in four weather windows — clear, rain, snow, night — on the same section and the same vehicle, so the comparison was fair.
Result: daytime accuracy increased from 82% to 96%, in bad weather and at night — from 61% to 89%. Misses decreased from 14 to 2 per 100 signs, false positives — from six to less than two per kilometer.
Stability was noted separately: during snowfall, object tracking no longer broke, and rollback to the previous model took one update cycle.
What to Do If the System Is Already Deployed on a Fleet
An existing traffic sign recognition system is rarely worth rewriting entirely — we start with an audit of the current model on recordings from your fleet.
We look at exactly where it makes errors: speed limit signs at twilight, temporary signs in a repair zone, sections with worn markings. We separately compare the dataset with real conditions — the model is often trained on clean daytime frames but operates on dusty optics.
The audit breaks down the pipeline layer by layer: frame capture, detection, route binding, event output to telemetry.
Often the cause of failures is not the model itself but time desynchronization between cameras, corrupted frames, or different optics calibration — this is fixed by configuration, without retraining and equipment replacement.
If accuracy is insufficient, we perform tuning without downtime: the new version runs in parallel with the current one on the same stream, discrepancies are calculated for each sign class, and switching occurs only after they fall within the agreed threshold. Rollback to the previous version is one step.
Fleet compatibility is addressed by groups: vehicle type, camera height and installation angle, seasonal dirt produce noticeably different quality. Frames from problematic groups go into targeted incremental training rather than being averaged across the entire stream.
Staged migration allows changing layers — for example, only detection while preserving the previous event output logic. Operations do not stop: we transition by vehicle groups, and the next one starts only after a cycle without degradation.
Technical Doubts Before Project Start
Questions at the start concern operating conditions more often than the model itself: what will happen in rain, at night, on worn markings, and during communication loss. Below are short answers to those asked before signing the work plan.
How does the system work at night and in bad weather?
Night, rain, and glare from headlights go into the training set as separate blocks, not as leftover frames. Each object receives a confidence score: a dark or blurred frame is marked as unconfirmed and sent for review rather than guessed.
What about worn markings?
The line is reconstructed from longitudinal edges, surface contrast, and the track from previous frames — short gaps and worn areas are covered. Where restoration is impossible, the section receives a lower weight in the calculation.
Can the model be incrementally trained on our data?
Yes. We label recordings from your sections, add them to the training set, and retrain. Classes on which the model already works remain in the set — otherwise quality on them will drop. Disputed frames are set aside until the next cycle.
What happens during signal loss or failure?
Recognition runs on the onboard computer, so communication loss does not stop it: frames and results are written to a buffer and uploaded when the channel returns. A separate process monitors the modules and restarts a frozen one — recovery takes seconds.
How do you check quality?
We build the test set from trips that were not included in training and calculate recall and precision for each class separately — day and night, in rain and on dry road. The discrepancy with the training set shows what data the model lacks.
Let's Discuss the Task and Come Back with a Work Plan
To come back to you with an implementation plan rather than general considerations, we need a minimum set of input data for traffic sign and marking recognition.
Below is what we request from the customer before start: this is enough to estimate the scope of work, risks, and provide realistic timelines.
- Task statement. Which sign classes and marking types are priority, whether only the fact of detection is needed or also condition assessment — wear, occlusion, partial erasure.
- Source materials. Video from real cameras at your sections, including difficult conditions: night, rain, low sun, dirty optics. From these we assess real accuracy, not laboratory accuracy.
- Acceptance criteria. On what share of frames we consider a detection correct, how we form the validation set, and what we consider a critical error — a missed sign or a false positive.
- Integration points. Where the result goes: map, dispatch center, onboard system, database. The export format and permissible latency are required.
- Timeline constraints. Whether there are hard dates — tests, stage acceptance, site launch — and whether the work can be split into queues by section.
- Access and site. Test environment where we deploy the model, rights to the recording data, and requirements for its storage.
Describe the task within these boundaries — we will come back with an estimate, stage plan, and what will be required from your side at each of them.
