Automatic product categorization: where manual sorting breaks down
When a catalog is measured in tens of thousands of product cards, manual categorization ceases to be a task that can be closed by people at all. Manual product labeling works only as long as the assortment grows slower than the team can process it.
After that, a queue appears: new items end up in sections by a formal criterion or by operator habit, rather than by the actual properties of the product.
The second failure line is rules. They accumulate over years: by brand, a product goes to one section; by a word in the title, to another; exceptions for specific suppliers are kept in a separate list.
There become more rules than items in some sections, they begin to conflict with each other, and no one can predict where a new product card will go. Thus catalog growth without rules turns into a set of local agreements held together by the memory of two employees.
Next, it is not the labeling that breaks, but the reason it was done. Properties are filled inconsistently: one part of the assortment has color and material specified, another has only a name.
Catalog filters show empty values, search finds an item only by exact match, and the customer does not reach a product that is physically in stock. A manager edits product cards on request, spending time on something that should have been resolved once at intake.
Errors in catalog sections cost more than it seems: part of the assortment simply does not exist for those who are searching, and any new category or supplier requires manual re-sorting.
We break down how this is closed — from order in the section structure to a model that labels incoming items and passes data to filters.
Implementation effect: catalog processing speed and share of manual edits
After the model is put into production, what changes is not a “feeling of order” but specific numbers.
The share of items that enter the catalog without human involvement reaches 85–95%, while the rest goes to a review queue — the operator sees not the entire stream, but only disputed cases.
Category matching accuracy on the validation sample stays above 95%, and this is not a one-time measurement but a metric recalculated on every upload.
Catalog processing speed is reduced from hours of manual sorting to minutes: the service labels the stream, rules build out attributes, and arrange items across branches.
New assortment becomes visible to the customer the same day, rather than after the operator works through the accumulated queue.
The unified category tree stops sprawling. The same category is no longer called differently by different suppliers, and the appearance of a new branch does not lead to manual edits of already processed items.
During seasonal spikes, when the flow grows several times over, time per item does not increase: the bottleneck shifts from people to the service throughput, which we set in advance.
To make the effect manageable rather than declarative, we configure the auto-acceptance threshold and a decision log: for each item it is clear why it was assigned to a category, and this can be selectively rechecked.
If a rule or weights cause a drop on new data, the pipeline returns to the previous configuration without stopping the catalog.
Labeling approaches: rules, learning from examples, language models
The first question at the start is what to use as the source of rules: dictionaries and rules, a model trained on labeled examples, or a language model.
The choice is determined not by the “modernity” of the method, but by catalog predictability — the stability of titles, the number of overlapping categories, and the frequency of new product groups appearing.
Therefore, we run the customer's history through all approaches and calculate the share of manual edits per thousand items.
In practice, a hybrid scheme more often wins: dictionaries cover explicit cases and serve as a safeguard against gross errors, the model resolves disputed cases, and the language model is connected where there is little labeled data. The human in the loop checks only low-confidence items — usually a few percent of the stream.
| Option | When it fits | Limitations |
|---|---|---|
| Rules and dictionaries | Stable catalog, formalized titles, dozens of categories | A new supplier wording breaks the rule; the dictionary grows faster than categories |
| Learning from examples | There are thousands of labeled product cards and a history of operator edits | A new category cannot be introduced without retraining; sensitivity to assortment changes |
| Language models | Little labeled data, categories described in words, product cards contain a lot of text | Instability on rare items; a confidence threshold is needed |
| Hybrid labeling scheme | The catalog changes, and there is an operator for disputed cases | A review interface and a “model — human” agreement metric are needed |
We fix the comparison criteria before the start: share of items without manual edits, accuracy on a holdout sample, and time to introduce a new category.
Implementation stages: from catalog audit to model handover
Categorization begins not with model training, but with a catalog audit: we export items with attributes, calculate the share of unlabeled items, duplicates, tree depth, and “junk” nodes.
Without this step, any quality metric hangs in the air — it is unclear what should even be considered the correct answer.
- Catalog and data audit — assessment of the completeness and uniformity of titles, attributes, and current categories. The output is a problem map: where data is insufficient, where labeling is contradictory, which nodes duplicate each other.
- Taxonomy schema — we agree on the category tree and rules for distinguishing close nodes. We fix which items are fundamentally ambiguous and remain for manual review.
- Sample labeling — a validation sample weighted toward rare categories, manual labeling, and measurement of agreement between labelers. This is the reference for all further measurements.
- Model tuning — a baseline on a pretrained model, comparison with keyword-based sorting, selection of confidence thresholds, work with imbalanced classes.
- Stream testing — a run through the real product intake pipeline in shadow mode: the prediction does not change the catalog, but every discrepancy is analyzed.
- Quality acceptance — the metric is calculated by class, not “on average”: a rare category weighs as much as a mass one. Items below the threshold go to manual review.
- Model handover — documentation, reproducible training, a procedure for retraining on new data, and analysis of typical errors.
After handover, the catalog is categorized in batches on your side, and the model is retrained on new items without our involvement.
Project artifacts: taxonomy, model, support procedures
An automatic categorization project ends not with a demonstration of the model on a holdout sample, but with the handover of a package your team can work with further without us.
Below is what remains with the customer: from the category tree to the rules by which the model is retrained.
- Category and attribute taxonomy — a tree with nesting levels, inheritance rules, and mandatory attributes for each leaf. The basis both for labeling and for export to accounting systems.
- Integration schema — where the model receives product data, in what form it returns the category and confidence, and what happens at low confidence. Two modes: batch catalog processing and product card processing at the moment of publication.
- Configured model — trained on your history, with fixed confidence thresholds and a list of categories it does not take and passes to a human.
- Model documentation — which features influence what, what data training was on, how to read quality by each category, and where the model systematically errs.
- Validation set — a fixed sample of products with reference categories. Running it shows whether the model has degraded after retraining.
- Support procedures — retraining frequency, who confirms new categories, how to roll back to the previous version if quality drops.
- Operator instructions — how to process disputed product cards and where to put corrections so that they enter the next training cycle rather than get lost.
Case study: a catalog of tens of thousands of items without manual labeling
The client is a retail chain, “Severny Dom”: a catalog of 120,000 items and 2,400 subcategories. Products were entered manually — a manager selected the section by title and description, and disputed cases went to a second round.
At the peak of seasonal supply, the labeling queue stretched for days, and some product cards were published in the wrong sections, so the customer simply did not find them.
Before
The share of incorrect section assignments was about 18%: almost every fifth product card ended up where it was not being searched for. A product appeared in the catalog two or three days after acceptance at the warehouse and was unavailable for order all that time.
More than half of the product cards required manual refinement: categorization rules did not cover new supplier lines.
What was changed and what the result was
The training sample was assembled from 8,000 items labeled manually according to a unified schema; the model learned from the title, attributes, and product image. The confidence threshold was set so that ambiguous items went to human review rather than being published at random.
Incorrect assignments in the stream — 3% instead of 18%. A product enters the catalog within an hour after acceptance: labeling starts automatically when the invoice is uploaded.
Manual refinement decreased from 60% of product cards to 8% — a human processes only disputed items and new lines that the model has not yet seen.
Once a month, accumulated corrections go into retraining, and we verify quality on a holdout sample — accuracy does not drop as the catalog grows.
How does rule-based categorization differ from a model trained on examples?
Rule-based categorization is synonym dictionaries, regular expressions, and attribute priorities: if a condition triggers, the category is assigned by a mapping table.
Such logic works from day one, requires no labeling, and is fully explainable: the log shows which rule triggered and why.
A model is a trainable classifier that looks at the title and attributes as a whole and generalizes patterns. It survives typos, new brands, and unfamiliar wording, but requires a cold start: several thousand labeled examples and validation on a holdout sample.
Explainability is lower — the output is a probability and the features that influenced the decision, not an explicit rule.
| Option | When it fits | Limitations |
|---|---|---|
| Rules and dictionaries | Narrow assortment, strict requirements for decision traceability | Manual refinement grows with every new category |
| Statistical model on attributes | There is historical labeling and a stable feature set | Tolerates new wording in titles poorly |
| Neural text classifier | Wide assortment, short and “dirty” titles | Requires a cold start and regular relabeling |
The approaches break differently. A dictionary has to be extended for every new category, and on a wide assortment, maintenance consumes more time than it brings benefit.
A model degrades imperceptibly: accuracy drops when suppliers and wording change, so the share of manual edits needs monitoring.
In practice, we assemble a hybrid: rules cut off unambiguous cases and filter noise in the data, and the model covers the rest.
Decisions with confidence below the threshold go to manual review and replenish the training sample — the share of automated processing grows from 40-60% at the start to 85-90% after several retraining cycles. For a quick start, a compact classifier such as FastText is sufficient.
Technical concerns: labeling quality, data drift, support
Technical objections at this stage rarely sound like “will the model work” — people ask about labeling quality, behavior on a live stream, and support when the taxonomy starts changing. Below are short answers for each of these points.
How much data needs to be labeled at the start?
The volume is calculated from the number of classes and their separability, not by the principle “more is better.” The practical minimum is several thousand examples per category with mandatory coverage of rare subcategories: a class shown by a dozen objects will simply not be identified by the model.
The rest is gathered through active learning — examples where the model's confidence is minimal go to labeling.
What to do when new categories appear?
We do not retrain the taxonomy entirely: we connect a new branch as a separate layer, relabeling only the disputed area. The boundaries of already working classes do not shift, so adding a category does not roll back quality on the others.
How does the model behave on a live stream — does it degrade?
We catch data drift not by overall accuracy, but by the share of examples with low confidence and the divergence of the input description distribution from the training sample, plus selective operator audits.
When the metric falls below the threshold, an alert triggers and a return to the previous version occurs.
How is this integrated into the current pipeline?
Integration is via a service interface or event queue, in batch or online mode. During the first period, the model runs in shadow mode alongside the current logic, so discrepancies are visible before switching, and rollback comes down to reverting the version.
Let's discuss the task: we'll come back with an estimate and implementation plan
After you reach out, we do not start with the model, but analyze the catalog: the depth of the category tree, the share of items with incomplete attributes, where the data comes from, and how often the section structure changes.
This analysis forms the basis for the categorization schema and scope of work — what we do and what will be required from your side.
- Project estimate. We calculate the catalog volume, number of categories, and assignment rules, and check whether there is enough data for quality control. The result is the scope of work and the order of stages.
- Pilot on a sample. We take several thousand items from different catalog branches and compare the result with reference labeling: we get the share of correct assignments and the share of items sent to manual review.
- Solution schema. We fix the confidence threshold below which a product goes to an operator, rules for new categories, and the procedure for processing items that fit several sections at once.
- Implementation plan. The order of connecting sources, quality control points at each stage, and rollback to the previous version of rules if accuracy on the stream drops.
- Handover to support. Documentation on rules and metrics, operator instructions, retraining procedures on new data, and analysis of disputed cases.
- Team training. We show how to read quality reports, change thresholds, and add categories without our involvement.
Describe the task and send a catalog export — we will come back with an estimate, schema, and implementation plan, including a pilot on a sample.
