AI system for detecting generated texts: where verification breaks
Student work verification is most often set up as a single request: the entire text goes to the service, the output is a generation probability, and the reviewer reads it as a verdict.
At the flow level for long works, this breaks: the text has to be cut into windows, the decision on a window does not equal the decision on the work, and the final result arrives without a breakdown by fragment — it is unclear which paragraph the conclusion relates to.
Mixed works are the most common case in practice: the student writes the introduction themselves, while the literature review is run through a paraphraser.
Generated-text detection gives them a borderline score, and two reviewers looking at the same number make opposite decisions — one rounds up, the other down.
If the university already has Turnitin or a similar service, a second verdict does not resolve the dispute but adds another discrepancy.
False positives finish off the procedure: established formulations, template phrases, translated sources, and texts by students for whom the language is not their first produce a consistently high score.
When a work is disputed, the department has neither fragments, nor trigger threshold, nor information about which model produced the result — there is nothing to close the appeal with, and academic integrity rests on the reviewer’s reputation rather than reproducible evidence.
We break down how this is solved engineering-wise: detection as a pipeline with paragraph windows, a single threshold, and a log of every decision, so that the verdict can be repeated and defended in an appeal.
What implementation delivers: verification speed and review of disputed works
Previously, machine-generation checking relied on selective review: an instructor physically could not read the entire flow and reviewed only individual works at random.
The AI detection system removes this fork — it labels the entire uploaded array, assigns each work a probability score for machine generation, and marks the fragments that contributed most to the result.
The instructor receives not a raw verdict but a ranked sample: at the top are the works that truly require attention, below are works with low probability.
The speed of work verification grows because continuous reading is replaced by review of one or two disputed fragments, and verification coverage becomes complete — the entire flow passes through the system, not individual folders and courses.
A unified scoring scale removes discrepancies between departments and subjects: identical probability thresholds, one rule for interpreting fragments, a fixed set of features.
Disputed cases stop depending on who exactly opened the work and how much time they had — the score is reproducible, and it can be explained to the student step by step.
Review of disputed cases is placed in a separate queue: next to the verdict are marker fragments, version history, and an edit trail, so the decision is made based on facts rather than impression.
Every manual decision is saved and becomes a basis for calibrating thresholds, so the share of works going to manual review decreases over time, and label accuracy grows.
The effect is visible in measurable terms: the share of works reviewed on time increases, the volume of manual review decreases, and the student gets a transparent appeal procedure with preserved rationale. Next, we explain how this works internally and what is needed from the educational institution.
Operating modes: batch checking, streaming, and integration into the workflow
The verification mode is chosen based on the volume of works, acceptable delay, and depth of integration into the academic process. Batch checking takes an array of works for a period and returns a report with a score for each document.
Streaming responds at the moment a work is submitted, while workflow integration returns the result directly to the learning system’s gradebook, without manual exports.
| Option | When it fits | Limitations |
|---|---|---|
| Batch checking via export | A semester’s array of works, re-review, appeals, and disputed cases | The response comes with a delay after submission; there is no reaction at submission time |
| Streaming check on submission | Online exams and time-limited works | Sensitive to peak loads during submission hours; requires capacity headroom |
| Workflow integration via API | Regular flow of works: scores and flags return to the gradebook automatically | Requires integration and a regulation — who reviews disputed triggers and by when |
| Mixed mode: streaming and batch | Draft signal at submission, refined review across the full array | Two rule branches and a shared threshold must be synchronized when changes occur |
Common to all modes are the limits of detection. No detector gives absolute accuracy: the most difficult class is mixed texts, where a person edited a draft or the model refined individual fragments, while short answers yield low reliability.
Therefore the system is configured not for a verdict but for prioritization: a threshold is set that separates signal from instructor decision, and the share of works sent to manual review is measured — that is the working quality metric.
Implementation stages: from work audit to handover of regulations
The quality of an AI detection system is determined not by the model but by calibration for a specific corpus of texts. Therefore the cycle goes from the archive of works to regulations: the result must be reproducible, not dependent on a single specialist.
- Work audit. We analyze the archive: assignment types, languages, traces of translation and edits. We identify classes of texts where detection is unreliable — we handle them procedurally.
- Pilot on a sample. We prepare a labeled set — machine, human, mixed, edited — and run the system blindly, without hints about the origin of the text.
- Verification scheme. We determine where the detector sits: at submission, during re-check, in appeal. The response format depends on the place — probability, fragment highlighting, or explanation.
- Threshold tuning. We select the boundaries “pass / manual review / return” for the acceptable false positive rate in your assignment profile.
- Quality control. We monitor metrics on the live flow: share of manual reviews, discrepancies with instructor decisions, behavior after version changes.
- Handover of regulations. We document thresholds, appeal procedure, actions on failure, and update rules, and train staff on real examples.
Each stage ends with an artifact: labeled set, verification route, threshold values, metrics report, instructor instruction. If thresholds need revision, you will know exactly at which step this is done.
What we hand over: models, labeling rules, documentation
We hand over not only the model itself, but everything by which verification is reproduced: labeling, thresholds, regulations, and decision log.
Acceptance is by checklist — for each artifact, you need to show that your team can repeat the result and analyze a disputed case without us.
- Model and threshold map — a working build plus a description of how the threshold affects the balance of false positives and misses, and which ranges are suitable for short answers, essays, and term papers.
- Data labeling — a corpus with labels and a labeling protocol: which features the expert considered evidence of generation, how they labeled mixed works, how they resolved disagreements between labelers.
- Classification rules — formal class criteria, a separate “requires manual review” zone, and interpretation of borderline values so the instructor does not decide this intuitively.
- Instructor regulation — a step-by-step scenario when a trigger occurs: recording the result, conversation with the student, the point where the academic office gets involved.
- Work storage policy — retention periods for texts and verification results, who has access, how a work is deleted at the student’s request.
- Documentation — data flow diagram, description of integration with the learning platform, model decision log, and reasons for each threshold change.
- Retraining instructions — how to add your own examples, which metrics to watch after an update, and how to roll back the model if quality declines.
The final step is acceptance: we go through the checklist together with your team, run control works, and confirm that every trigger is explainable. Describe the task — we will return with an assessment and a handover plan.
Case study: accuracy of application review in the admissions committee before and after
Admissions campaign at a large university: several thousand applications per season, each with a motivation letter and essay. Some texts were written by a generative model, and the committee caught this “by eye” — by smooth formulations and suspiciously even structure. The verdict depended on which expert read the work.
Before implementation, a repeat review of the same work by another expert changed the decision in roughly every fourth case. Reviewing one application took about 20 minutes, and the share of appeals remained high.
What changed: detection works not on the whole document but on fragments — the system marks specific paragraphs and shows confidence for each.
The trigger threshold was configured for the committee’s regulation: clear cases go into the report automatically, disputed ones go into a separate manual review queue with highlighting. The final decision on the application remains with a human.
What fragment-based review delivered
After: disagreement between experts dropped to 8%, the share of appeals became three times lower, and the average review time for one work fell to 4 minutes.
We separately checked false positives on texts by non-native authors — this group was most often suspected; the risk is also confirmed by external analyses, including Turnitin.
The main class of errors that was eliminated was accusations against applicants with a formal or template writing style. The committee sees not the label “generated” but specific supporting fragments, and the appeal is reviewed based on them, not on overall impression.
Text detection or plagiarism checking: what is the difference
These are different tasks, and they are solved by different methods.
Plagiarism checking answers the question “does this fragment exist in a known source,” while generated-text detection answers “was it written by a model.” The first works with a source database, the second with properties of the text itself.
Plagiarism search splits a work into short word sequences and looks for their matches in an index: articles, term papers, web pages. A match above the threshold is marked as borrowing, everything else is considered original. The approach catches copying well, but says nothing about the origin of the text.
Generated text almost never matches a source verbatim: the model builds new formulations rather than copying a corpus. A work paraphrased through a generator will receive a low plagiarism percentage — even though it contains no original contribution. Here plagiarism checking is silent by design, not because of a threshold misconfiguration.
Detection relies on other features: word predictability, variation in sentence lengths, repetition of constructions, evenness of style throughout the document.
A model produces statistically smoother text than a human, so the decision is made based on a probabilistic score, and the result is expressed as the share of generated fragments, not as a percentage of matches.
In practice, mixed works are more common: the thesis and conclusions are written by a human, while the literature review is produced by a model. Therefore the score is calculated by paragraphs, and the interface shows specific fragments with features that can support review.
The instructor sees not a single number but a picture across the document — this is also required for a conversation with the student.
Technical concerns: false positives, languages, load
The detector’s verdict is not a binary “generated / not generated” but a confidence score with highlighting of feature fragments.
False positives occur where the threshold is set crudely: template phrasing, a textbook quotation, a pre-memorized answer — all of these are statistically similar to machine text.
Therefore we calibrate the threshold on the customer’s real student works, not on a synthetic set, and give the instructor not a label but a basis: which sections triggered and with what strength. A disputed work can be manually rechecked without rewriting the entire verdict.
Multilingual texts are analyzed on a single scale: a mixed Russian-English paragraph, an insertion in another language, or code-switching within a sentence do not break the score.
We separately check behavior in each campus language — otherwise the system confidently errs on a rare language while working normally on the main one, and this is the worst class of failure.
What happens during peak load — exams, a flow of works at the end of the semester?
Analysis runs as a prioritized queue: single works are processed interactively, batch uploads in the background, and a long batch does not block an individual student.
Processing nodes scale horizontally, state is not stored on a specific node, so adding capacity does not require rebuilding the system. Under overload, the service degrades predictably: secondary checks are disabled first, but the verdict and queue are not lost.
Where are works stored and who sees them?
Storage is configured according to the customer’s regulation: text lifetime is set by policy, and after it expires texts are deleted automatically, not “sometime.” Access is role-based — an instructor sees their groups, a department administrator sees a summary, no one gets access to another’s flow.
Depersonalization disables linking a work to a student where statistics are sufficient. We agree on data placement before launch, including options with storage inside the institution’s perimeter.
How can we make sure the system is not wrong before integrating it into the academic process?
Before launch, we run the detector on an archive of reviewed works: some with known authorship, some with confirmed generation. This gives the false positive and miss rates specifically for your discipline and your language, not a market average.
Then the threshold is adjusted, and you can return to the same set after a model change — the numbers are comparable.
Let’s discuss the task: which works we check and at what volume
At one university, checking relied on manual review: the instructor opened the document’s edit history and issued a verdict themselves. We split the flow into two levels — automatic fragment labeling with generation probability and a separate queue for disputed cases.
The share of works sent for a manual verdict decreased from 40% to 6%, and the student’s response began arriving the same day instead of several days later.
This is sustained not by a “sensitivity threshold” but by configuring the verification workflow to the rules of the specific educational institution.
To discuss the task substantively and return with an assessment and plan rather than general words, we need the verification volume in numbers:
- types of works: essays, reports, term papers, theses, lab reports;
- input formats: text files, PDF, typing in the editor;
- upload channels: student portal, group export, integration with Moodle;
- verification languages and the share of mixed texts;
- requirements for false positives and the student appeal procedure;
- archive of past checks — we will validate quality on your own works.
Next we set the timeline: what we do on our side, what is needed from you, when we launch the system for one faculty in trial mode, and in which week we compare metrics.
As an output, you receive a work plan with stages and areas of responsibility, not a retelling of capabilities.
