AI system for detecting plagiarism and cheating in education

Developing an AI plagiarism detection system: where accuracy breaks down

Classic plagiarism checking compares text by n-gram overlap and confidently finds direct copies—but it breaks down where a student has paraphrased a paragraph or run it through back-and-forth translation.

Lexical hashes yield almost zero match, the work is classified as "original," even though the source is reproduced in meaning.

The flip side is quotes, definitions, standard formulas, and bibliographies: they match across hundreds of papers and pull overall similarity up.

The instructor receives two conflicting signals at once: an obvious borrowing is missed, while honest work is flagged as plagiarized. After two or three such reviews, checking becomes a formality—it is skimmed rather than used as an argument in a conversation with the student.

Technically, this is a conflict of metrics, not a "bad algorithm." Lower the fragment similarity threshold—recall grows, but the share of false positives on quotes and formulas soars. Raise it—paraphrases and translated borrowings are lost.

The task cannot be solved with a single number, so the assessment is built in layers: exact match, semantic similarity via vector representations, signs of machine translation, and separate filtering of quoted and reference material based on document markup.

This works only with calibration on a labeled set of papers from a specific discipline: an essay in history and a physics lab have their own thresholds and their own exclusion rules.

Below, we break down what such a system consists of and how instructors' trust in its verdicts is built.

What an implemented plagiarism checking system delivers

We measure implementation metrics not by the number of "matches found," but by four indicators that a department head sees: recall in identifying disputed works, the share of false positives, the time an instructor spends reviewing one paper, and the size of the manual sample needed to quality-control the system itself.

Detection recall grows because checking is done by fragments rather than by the whole file: the system compares paraphrase, changed sentence order, and borrowings from papers from previous years, not only verbatim copies.

Thresholds are set separately for essays, term papers, and calculation assignments—so correct quotes do not turn into a stream of false positives, and review remains meaningful.

We also check against external corpora: databases of scholarly publications, open sources, and services such as Turnitin, if the university already relies on them.

For the instructor, this changes the nature of the work: instead of continuous comparison, they look at a report where disputed places are marked together with the source and match percentage.

The manual sample is reduced to a control share—enough to check the quality of the system itself and threshold tuning, rather than replacing it with manual labor.

Disputed papers go to review with the report saved, and an appeal can be conducted based on it without going through the text again.

Comparison of approaches to plagiarism and machine text checking

Checking schemes differ not in model quality, but in the point at which a paper enters analysis and how many false positives the customer is willing to accept. Therefore, approaches should be compared by flow coverage and by the impact of an error for a specific type of work.

In practice, we more often assemble a combined scheme: a quick filter at submission cuts off obvious matches, while a regular batch run over the archive catches cross-borrowings and papers written by machine text.

We document the limitations of each option in the checking procedure—then disputed triggers are reviewed according to pre-agreed rules rather than manually.

Implementation stages: from auditing the paper corpus to handover

Implementation is divided into stages, each ending with an artifact that is verified before moving to the next.

The order is strict: first we establish what exactly counts as a violation, and only then do we configure automation—otherwise thresholds are fitted to convenient examples and the system falls apart on real papers.

1. Audit of the paper corpus. We analyze papers verified by instructors by assignment types and formats—text, code, formulas, drawings. We look at how references are formatted and where correct retelling turns into borrowing.
2. Flow diagram. We establish where a paper comes from, where it is stored, and in what form it reaches the reviewer; what to do with drafts, retakes, and group projects—otherwise duplicates distort statistics per student.
3. Citation rules and thresholds. We configure what quote length is considered correct, how reference formatting is recognized, and where the boundaries between genres lie: a term paper, an essay, and program code are assessed by different rules.
4. Calibration on disputed examples. We collect borderline cases where the decision is not obvious and compare the system's assessments with instructors' decisions. We analyze discrepancies and adjust settings until disputed papers begin to be interpreted consistently.
5. Testing on a holdout sample. We run the system on papers that did not participate in tuning and look at recall and precision for each assignment type: where a borrowing was missed, and where a correct quote was flagged.
6. Handover and training of reviewers. We provide a description of the rules, an analysis of disputed cases, and the appeal workflow; reviewers learn to read the report and understand what each label is based on.

Handover artifacts: configurations, diagrams, and procedures

After launch, the system lives without us: methodologists configure it, instructors use it, and the quality service checks it.

Therefore, together with the working service, we hand over a complete set of artifacts—using them, the system can be operated, further tuned, and transferred to another team without contacting the authors.

• Paper checking configuration. Separate profiles per assignment type—term paper, lab work, coursework, exam answer: similarity thresholds, the set of enabled checks (internal corpus, open sources, overlaps between papers in the same flow), rules for excluding quotes, title pages, and commonly used phrases.
• Paper flow diagram. A diagram of the path from file upload to result card: which services are involved, where originals and intermediate snapshots are stored, what happens when the learning platform is unavailable, and how the queue is recalculated.
• Integration diagram with the learning platform. Call points (upload, grading, appeal), exchange format, behavior on timeout and request retry, so that a duplicate submission does not create a second check.
• Reviewer access policies. Roles—instructor, methodologist, administrator, appeal review: what each sees, who has access to full texts and who only to the report, retention period for student papers, and access log.
• Dictionary description. Subject-specific dictionaries of terms and set phrases that reduce false positives on formulas, template phrases, and quotes; rules for updating and versioning so that editing a dictionary does not break historical assessments.
• Procedure for reviewing disputed cases. Who makes the decision, what evidence is attached (fragments, sources, match share), within what period the student receives an answer, and how changes to the procedure are transferred into the checking configuration.
• Documentation and acceptance. Instructor guide, instructions for updating dictionaries and profiles, deployment diagram, acceptance checklist on a labeled set of papers—with it, quality after any further tuning is confirmed reproducibly rather than by eye.

Case study: how false positives were reduced in a checking flow

The customer's checking flow has several hundred student papers per week: essays, reports, lab work.

Comparison was done through the external service Turnitin and an internal archive of previous years, but the result was dumped into one manual review queue without separation by assignment type.

What it was like before tuning

The similarity threshold was the same for all papers, so about 40% of the flow went to review. Title pages, bibliographies, and standard problem statements produced high similarity and looked like violations.

An instructor spent 12–15 minutes on one disputed paper, and a violation was confirmed in fewer than half of cases—the rest was noise.

What was changed and what the result was

We separated thresholds by assignment type and added exclusion dictionaries: normalized fragments are filtered out before comparison.

We introduced a cascade—first automated checking against the archive, then human review, with a disputed case marked as "needs a dialogue with the student" rather than as a ready accusation.

The false positive rate fell from 40% to 8–10%, review time per paper dropped from 12–15 to 4 minutes, and about 15% of the flow is reviewed manually.

Confirmation of violations in this sample increased roughly twofold: the reviewer sees disputed texts rather than a stream of formal matches. Each decision is logged with a reference to the fragment—the appeal is reviewed using the saved trace.

How plagiarism search differs from machine text recognition

Borrowing and machine generation are two different tasks, and they are solved with different mathematics.

Plagiarism search is comparison against a corpus of sources: the paper is split into fragments, each is searched in the index, intersections are found, and their weight relative to the text volume is calculated. The decision here relies on the fact of a match, not on the author's style.

Assessment of machine generation markers works without a corpus. The model looks at the statistics of the text itself: uniformity of sentence length, low lexical variability, predictability of transitions, scarcity of rare phrases and typos.

This is a probabilistic quantity—it answers not the question "was it copied" but "does it look like model output."

Hence the different decision thresholds. For borrowings, the threshold is set by the share of matched fragments and tuned by discipline: in legal papers, quotes are the norm, so correctly formatted references are excluded from the count and reduce false positives.

For machine generation, the threshold is probabilistic; it is calibrated on a labeled sample from instructors to keep the false accusation rate acceptable.

A combined assessment of a paper does not add two numbers into one, but builds a profile: the share of borrowings by section, the distribution of suspicious sections across the document, and generation markers by paragraph.

The decision is made by a human—the system shows which fragments should be opened and what to compare them with, and saves the markup for appeal review.

The errors in these loops are also different. A false positive for borrowings is treated by excluding the source from the index or refining citation rules; for generation—by recalibrating the threshold for the discipline and genre.

Therefore, in the reviewer interface, these are two independent indicators, not a single "originality" score.

Technical concerns: thresholds, dictionaries, retraining

The concerns are not about the model itself, but about its integration with the academic process: how to update dictionaries, what to do with an already working checking system, and how not to break the instructor's usual scenarios. We address what most often comes up during the approval stage.

We do not take "market-average" thresholds: we calibrate on conflicting pairs from your discipline—correct quotes, formulas, typical paraphrases. We keep benchmarks such as Turnitin as a reference point, but the threshold is chosen based on your sample.

You already have a checking system deployed—does everything need to be rewritten?

No. We connect to its results and first run in shadow mode: we compute verdicts on historical papers and analyze discrepancies manually. Only after that do disputed groups of papers move to the new logic—without stopping checks.

How is checking integrated into the learning platform?

Through an API: the paper is sent for analysis asynchronously, and the result is returned to the assignment card with highlighted matched fragments and a link to the source. The instructor sees the usual screen; only the source of the verdict changes.

What to do with disputed citation cases?

The model does not penalize a correctly formatted quote—it distinguishes it from a paraphrase, even if the wording is close to the source. Borderline cases go to the manual review queue with an explanation: what matched with what and why.

Let's discuss the task: what to clarify before estimation

To come back with an estimate and an implementation plan rather than general wording, we need a snapshot along four axes: what and in what volume we check, how the paper flow is structured, who makes the decision on disputed cases, and what the system will need to be integrated with. Below are questions that remove most of the uncertainty before the start.

• Corpus volume. How many papers per year are checked and in which languages; whether there is an archive of previous years for comparison and in what form it is stored—scans, editable documents, exports from the learning system.
• Checking flow. At what point detection is enabled: on paper upload, as a scheduled batch, or at an instructor's request. This determines whether a response within seconds is needed or whether a deferred queue is acceptable.
• Reviewer composition and rights. Who sees the report—instructor, methodologist, academic office; whether visibility restrictions by their own groups, an action log, and separation of access to students' source papers are needed.
• Integrations. What is already in use—learning portal, file storage, email, internal authentication. The availability of APIs and single sign-on determines whether this will be a manual export or automatic exchange.
• Markup rules. What is considered acceptable borrowing—quotes, template phrases, standard problems; how violations are currently recorded and who approves the verdict in a disputed match.
• Data requirements. Where student papers are allowed to be stored, whether there are restrictions on external services, and whether a local environment without texts leaving it is needed.

After the answers, we finalize the solution design and come back with an estimate and a stage-by-stage implementation plan.