Aspect-based sentiment analysis: what the average score hides
The average score of 4.2 looks healthy exactly until you look at the distribution. Within a single rating live opposite signals: the courier arrived on time, but the components had to be replaced twice.
The customer gives four stars, the team reads "generally fine" and touches neither logistics nor assembly — that is why the average rating distorts the picture, and decisions based on it are made at random.
Aspect-based sentiment analysis removes this limitation at the model level: each review is decomposed into semantic topics — quality, price, support, convenience, delivery — and a separate polarity is calculated for each.
The same text can be enthusiastic about the service and sharply negative about the packaging, and this is no longer averaged into a single number.
The bottleneck of manual review of reviews is that it does not scale. An analyst honestly labels several hundred texts, and then the choice begins: either lose depth and count by stars, or drown in volume.
Product feedback arrives with a delay of weeks — by the time of the report, the reason has already spread across reviews and write-ups, and fixes have to be made following a cold trail.
Aspect-based review analysis changes the very rhythm of work: a fresh batch of feedback is labeled in hours, not weeks of manual review.
The product receives a signal on a specific topic — "assembly," "payment," "warranty" — instead of an overall score for everything at once. Next, we break down how this works in practice: from data preparation to a regular report you can rely on.
What changes after implementation: effect in metrics
The effect of review analysis is measurable in five values: share of labeled corpus, response time to negative feedback on a specific aspect, manual workload removed, completeness of detecting rare aspects, and dynamics of product quality metrics.
Speed of processing the review stream
Previously, the stream was analyzed selectively: the analyst had time to go through part of the corpus, the rest waited in line.
Aspect-based sentiment analysis on GPT-class models processes the entire volume, so a complaint about "delivery," "interface," or "support" reaches the report the same day, not after manual sampling.
- Share of labeled reviews — up to 100%: short and ambiguous texts no longer fall out of the sample.
- Response time to negative feedback by aspect — from weeks to one day: a drop in score is visible in dynamics, not after the fact.
- Manual labeling is removed: the analyst checks model discrepancies rather than labeling the corpus from scratch.
What stops being lost during manual labeling
People gravitate toward frequent and explicit topics: rare aspects and mixed reviews, where one phrase praises and criticizes at the same time, fall out of labeling.
Aspects with a declining score live within the margin of error for months and surface when product quality metrics have already dropped.
- Rare and indirectly worded aspects are captured on par with high-volume ones — the class of omissions goes away.
- Each aspect gets a history: you can see when the score started going down, not just the final drop.
Approaches to review analysis: comparison of operating modes
Comparison of approaches to review analysis comes down to three modes, and the choice between them is determined not by fashion but by the stability of vocabulary in the domain and the volume of labeled examples.
All three produce aspect sentiment like "delivery is fast, packaging is crumpled," but they handle slang, typos, and mixed-sentiment reviews differently.
When rules and dictionaries are enough
The dictionary method is lists of terms, intensifiers, and negation handling rules. It is explainable: you can see which rule produced the label, no labeling is needed at all, and the load is predictable.
This is enough when reviews are of the same type, there are about ten aspects, and vocabulary changes slowly.
| Option | When it fits | Limitations |
|---|---|---|
| Rules and dictionaries | Narrow domain, stable vocabulary, need explainability of each label | Does not handle irony and slang; dictionaries are maintained manually |
| Trained classifier | Large stream of similar reviews, there are labeled training examples for the required language | Will not see hidden aspects outside the labeling; when vocabulary changes — retraining |
| Large language models | Free-form text, multiple languages, rare aspects, little or no labeling | Need a response schema and result validation; load constraints at volume |
Where large language models are needed
The model reads the review as a whole: it identifies aspects that were not in the list, ties the sentiment to the right mention, and transfers the logic to a new language without separate labeling.
Instead of thousands of labeled examples, a dozen reference ones plus a strict response schema are enough, and discrepancies are caught by checking against a control sample. The price of flexibility is instability of wording and mandatory validation of the analysis.
In practice, the modes are combined: dictionaries and the classifier cover high-volume, similar cases, while the model covers rare aspects and free-form text.
How we implement: stages from data audit to handover
Implementation of review analysis begins not with model setup but with data: without an audit of the corpus, automatic metrics look convincing but diverge from reality. Below is the sequence by which we lead the project through to handover and training of your team.
Data audit and aspect schema
-
Audit of the review corpus. We count the volume and languages, the share of duplicates from different platforms, separate out noise — emoji, transliteration, form fragments. The result is a data map: what is analyzed automatically and what we clean at input.
-
Aspect and subtopic schema. We build a tree: a top-level aspect (delivery, product quality, support) and subtopics within it. We agree the schema with you so that each branch answers a management question.
-
Extraction setup. The model of the current branch receives the schema, synonym dictionaries, and examples of borderline wording. We fix the rule for reviews without an explicit score separately: "not mentioned" does not equal "neutral."
-
Labeling quality check. On a blind sample, we compare the model's answers with human labels in Label Studio and calculate discrepancies by aspect and sentiment class. We refine the rules until stable agreement — otherwise automation breaks on the first release.
Rollout and validation on a live stream
-
Review stream processing schema. We connect sources and a queue: new reviews are analyzed incrementally, with the source text and schema version preserved — this ensures reproducibility during revisions.
-
Threshold calibration. On the live stream, we select the confidence boundary and send disputed cases for manual refinement. The share of automatically analyzed reviews grows, and errors on rare aspects become fewer.
-
Handover and team training. We hand over configurations, documentation on the aspect schema, and retraining regulations, and conduct training — you make changes on your own.
What the customer receives: configurations and regulations
Along with the working system, we hand over artifacts by which it can be developed without us: aspect descriptions, schemas for interaction with review sources, rules for handling data, and quality check procedures.
Each document has a purpose — add a category, connect a new channel, or resolve a discrepancy in scores.
Documentation and schemas
- Aspect map: ontology of categories with boundaries, examples of borderline reviews, and rules for interpreting mixed sentiment.
- Integration schemas: where reviews come from, how re-ingestion, deduplication, and export of results to accounting systems are arranged.
- Documentation on review analysis: what confidence by aspect means, where there is neutrality, and where there is a lack of data.
- Data access policies: roles, anonymization before entering storage, access log, retention periods.
- Configurations: dictionaries, post-processing rules, and confidence thresholds are moved into separate files, not code.
Operating regulations
- Labeling quality control: manual sample check, measurement of annotator agreement, actions when accuracy falls below threshold.
- Instructions for new categories: aspect description, set of examples, run over historical reviews, comparison with the current version, rollback if it worsens.
- Ontology versioning: each change has its own version, old results remain reproducible.
Case study: how a blind spot in delivery reviews was removed
The client is a ready-to-eat food delivery service Ложка: several tens of thousands of reviews per month from the app, aggregators, and social networks.
Before implementation, any review was reduced to a single rating — courier, packaging, dish temperature, and order composition were mixed into an overall score.
A failure on a separate aspect, for example packaging coming apart, dissolved into an average three and did not reach the responsible person.
What it was before and what it became after
Before, analysis was manual: about a tenth of the stream was reviewed, the rest was not analyzed, and reaction to negative feedback took up to two days.
After, the model labels each review by five aspects — courier, packaging, temperature, order composition, app — and separately evaluates the sentiment of each. The share of analyzed reviews grew from 12% to 96%.
A failure on a separate aspect is now visible the next day: for packaging, the share of negative feedback grew fourfold in a week, although the average order rating dropped by only 0.2 points and was not visible in the overall picture.
A threshold rule by aspect sends a notification to the responsible person, and reaction time to negative feedback decreased from two days to three to four hours.
Disputed labels — about 5% — go to manual review, so by aspect you see not "mood" but a concrete cause: some restaurants changed packaging, and the share of negative feedback on this aspect returned to its previous level within three weeks.
How to calculate sentiment if reviews are in multiple languages?
We calculate not the sentiment of the review as a whole, but the sentiment for each aspect — delivery, product quality, support performance.
For multilingual reviews this is fundamental: aspects in different languages are described by different words, and a mixed review cannot be reduced to a single label.
The analysis schema comes down to three options, and each has its own cost in terms of robustness and maintenance.
| Option | When it fits | Limitations |
|---|---|---|
| Single multilingual model | stream in 10+ languages, each gives a small share | degrades on rare languages and mixed reviews; retraining affects all languages |
| Translation before analysis | 2–3 main languages, need common rule logic | translation loses slang and negations, adds latency |
| Separate pipelines by language | one or two languages give the bulk of reviews | more expensive to maintain: own rules, dictionaries, and tests for each |
In practice, we set up routing: language detection at input, a single model as the base, and separate pipelines for languages with a large share and weak performance of the base model.
We analyze mixed reviews segment by segment: we split the text into sentences and send each fragment to its own pipeline.
Control is on a labeled sample for each language: the share of found aspects and the share of correctly determined polarity. It is visible which language pulls the overall metric down and requires a separate pipeline.
Technical doubts: accuracy, labeling, retraining
Engineering objections to aspect-based sentiment analysis come down to one thing: where is the guarantee that the model will not invent an aspect and will not fall apart on new categories.
We address the four doubts below before the start — the labeling schema and rollout order depend on the answers.
How is the accuracy of aspect labeling checked?
A control set of 300–500 reviews is labeled by two independent annotators, and discrepancies are resolved into a single schema. The key metric is inter-annotator agreement: low agreement means blurred aspect boundaries, not a weak model. Each prompt variant is checked against this set.
What to do without historical labeling?
We cover a cold start with weak labeling: rules and dictionaries by category give draft labels, which we correct selectively. For the first weeks, the model only proposes aspects, the decision remains with a human, and their corrections become the training set.
How is quality drift on new categories caught?
Once a week, a random sample goes to manual review. Separately, we count the share of aspects that are not in the reference book — this is how invented aspects are controlled. If the share grows, a new type of complaint has appeared; we expand the reference book and the prompt.
What about short reviews and typos?
The bulk of errors are short reviews and texts with typos: in three words, an aspect is often not explicitly expressed.
We calculate accuracy on them separately, and if it is below the overall accuracy, we lower the model's confidence and send the review for human review rather than guessing.
Let's discuss the task: we'll send an estimate and work plan
To come back with an estimate and work plan, we need a minimum of input data: an export of reviews as is — with typos, emoji, mixed languages, and duplicates; a list of aspects that are important to separate ("delivery," "assembly quality," "support performance"); constraints on the data perimeter — whether we run on your hardware or calls to an external model are allowed.
If there is no aspect dictionary, we will assemble it from the corpus and agree it with you on a labeled sample.
In response, we will send the scope of work by stages with estimated timelines and acceptance points.
Separately, we will show the processing schema: where the model extracts aspects and sentiment, how much manual labeling is needed for quality control, and how agreement with expert labeling is calculated.
- Processing schema: review ingestion, cleaning, aspect extraction, sentiment, aggregation by products and channels.
- Scope of work by stages, dependencies between them, and acceptance points.
- An agreed aspect dictionary and rules for analyzing disputed wording.
- Requirements for the data perimeter: where texts are stored, who has access, what is logged.
- Quality metrics: share of correctly extracted aspects and sentiment accuracy on a control sample.
- Plan for putting into operation: a one-time run over the archive, then regular updates.
- List of risks: what the model will not see at the current data volume.
Describe the review analysis task — send examples and specify constraints on the perimeter. We will come back with an estimate, plan, and scope of work; the project can start immediately after they are agreed.
