AI system for automated journalism: generating news from data

Automated journalism: where the manual pipeline breaks

Automated journalism begins where the flow of events arrives faster than the newsroom can process it.

Data updates continuously — quotes, statistics, results, logistics — while manual news preparation depends on a person: they open the source, compare snapshots, transfer numbers into text. While the material goes through the chain, the next data snapshot already changes the picture.

Hence the main cost is publication delay. The material comes out hours after the event, when the window of demand has already closed, and the text itself describes a state that no longer exists.

At the same time, every update requires a new edit, and with high-volume flows the newsroom either loses some events or publishes them late and with outdated numbers.

The second class of failures is duplicates and discrepancies in data. The same fact comes from several sources, reaches different authors, and turns into two materials with mismatched numbers.

The discrepancy accumulates unnoticed: different sampling period, different aggregation currency, different rounding rules. The reader sees two texts about the same event with different numbers, and trust in both falls.

Trying to solve this with “write faster” does not work: the bottleneck is not the person’s speed but the scheme itself — a manual pipeline does not scale with the frequency of events and does not maintain a single source of truth for numbers. We’ll break down how this is solved.

What automated journalism gives the newsroom and the product

Automated journalism works where there is a flow of structured data: quotes, statistics, match protocols, agency reports.

The pipeline turns them into a draft, and the editor focuses on meaning rather than transferring numbers into a template — below are the effects visible in the newsroom’s metrics.

Publishing speed and topic coverage

Material that previously waited for an available correspondent is published minutes after the source updates: the delay is determined by the verification procedure, not by staff workload.

A template with variables and substitution rules covers typical storylines — local statistics, round results, indicator dynamics by district.

Coverage grows without loss of quality: where the newsroom wrote about the ten largest cities, the pipeline produces a draft for all of them, and an anomaly filter surfaces what truly deserves attention.

Transparency of sources and eliminated errors

Every statement in the draft is linked to a source data field with the time received: during review, it is clear where the number came from and which rule brought it into the text.

This closes a class of errors that in manual work is caught late — a mixed-up unit, an outdated snapshot, values for the wrong period.

Edits shift to style and headlines rather than facts: the share of materials reaching publication without substantive edits grows, and the newsroom’s time goes to topics that need a live reporter.

This order of effects is also documented by industry reviews — Review of practices for automated text generation from structured data.

Generation modes: template-based, hybrid, and model-based

The generation mode is chosen not by taste but by the structure of the input data and the cost of an error.

The more predictable the input signal, the more rigidly the wording can be fixed; the more heterogeneous the sources and the more complex the storyline, the more freedom the model needs. We configure the mode for the flow, not the other way around.

When template mode is sufficient

If data arrives according to a fixed schema and there are dozens of events of one type per day, the template gives a predictable result: validation rules cut off broken records before publication, and the discrepancy in wording between outputs is zero. Replacing the template or adding a new event type is a configuration edit, not retraining.

When hybrid mode is needed

A hybrid is used where the model writes coherent text, but numbers and proper names are substituted from a verified source, and disputable points go to the editor.

This preserves publishing speed while maintaining control over facts — we limit automation to topics where the cost of an error is lower than the cost of verification.

Implementation stages: from data audit to publication

Automated journalism is not a model connected to a news feed but a pipeline where an error in input data becomes a factual error in publication.

Therefore, the work proceeds in stages: first we analyze the data and rules, then we achieve predictable text quality, and only after that do we put the flow into production mode.

Preparation and source audit

1. Data source audit. We inventory feeds, exports, and internal storages: completeness of records, event appearance delay, duplicates, format stability. For each source, we mark fields suitable for machine processing.

2. Entity schema. We consolidate disparate records into a single model: people, organizations, places, events, and numeric indicators. Without it, the template substitutes a number for the wrong subject — a typical cause of factual errors.

3. Event selection rules. We set significance thresholds, deduplication, and priorities: what goes into generation, what is merged, what is filtered out as noise. We agree on the rules with the newsroom rather than deriving them automatically from the data.

Fact-checking and acceptance

• Generation setup. We assemble templates for each event type, set length, tone, and variability of wording so that the flow does not look cloned.

• Text quality acceptance. We run the system on historical events and verify every number, name, and date against the source. Separately, we calculate the share of publications that went out without editor edits.

• Production launch. We move the pipeline to a schedule, add delay monitoring, a reprocessing queue, and stopping of output when there is a discrepancy with the source.

• Handover and training. We provide documentation, an edit procedure, and control points so that the flow can be maintained without our involvement.

What the client retains after implementation

After project delivery, the client’s environment retains not only a working service but also the entire set of settings by which it can be developed, handed over to a shift, and checked.

Below is what is recorded in the documentation and repository rather than left in correspondence with the team.

• Generation configurations: request templates for each genre — news, digest, data summary — rules for length, headline structure and mandatory blocks, thresholds for wording variability.
• Source and relationship schema: list of connected data channels, priority between them, polling frequency, deduplication rules, and signs by which a record is considered unreliable.
• Templates and tone rules: dictionaries of allowed and prohibited wording, stop lists, rules for citing and disclosing the data source.
• Access policies and roles: who edits templates, who publishes, who only reads the log; separation of rights between newsroom, analytics, and contractor.
• Publication procedure: the path of material from data arrival to publication, points of mandatory human review, conditions for automatic publication, and the recall procedure.
• Publication logs and instructions: who published what and on the basis of which data, what edits the editor made, how to connect a new source, and what to do when a channel fails.

Procedures and documentation

Documentation handover goes together with training: we run through scenarios with the newsroom and engineer — connecting a source, rolling back a template, analyzing an erroneous publication.

After that, the team changes generation configurations and tone rules on its own, without contacting the developer.

Separately, we define what counts as an incident, the timeframe for analyzing it, and the metrics by which publication quality is checked. This keeps the system manageable when it is already being developed by the internal team.

Case study: a news flow from open data instead of a manual feed

The project is a news generator from open data: the service itself pulls updates, checks them for anomalies, substitutes values into a template, and places the finished draft in the editor’s queue. The source is open statistics portals, based on Unified Open Data Platform. Below are metrics before and after moving the feed to such a flow.

What it was before

The duty editor assembled the feed manually: downloaded the export, compared indicators, wrote the text, and filled out the card. One material took about 40 minutes, 8 issues were published per day, and the feed went silent by evening.

During manual verification, 12% of publications had discrepancies in numbers — they were taken down or corrected after publication, and every fourth material was returned for revision. Duty meant constant presence at the screen.

What it became after

Data arrives as a stream, anomaly checks and value verification are performed before entering the template, and the editor sees a queue of finished drafts.

Material preparation dropped to 2-3 minutes — mostly for proofreading wording, and the number of issues grew to 60 per day without expanding the shift.

The share of publications with discrepancies in numbers fell below 1%: values are verified against the primary source automatically. Returns for revision are about 5%, and the duty editor’s workload has been reduced to a one-hour check of the queue.

The feed no longer depends on one person: issues are published on weekends too, and the editor is occupied with meaning and wording rather than transferring numbers.

How does template-based generation differ from model-based generation?

Template-based generation is the deterministic substitution of data into pre-written constructs: the same news item with the same input numbers is assembled into the same text.

Predictability here is maximal, verification comes down to checking substitutions, and any deviation of input data from the expected format is detected immediately — either by skipping the output or by an explicit error in a field, rather than a plausible fabrication.

The price for this is that wording variability is almost zero: across a series of same-type events, the reader sees the same phrase, and the feed begins to look mechanical.

The model works the opposite way: it rebuilds the wording for a specific event and maintains coherence on non-standard storylines, but its errors are of a different class.

Instead of a refusal, it produces confident text with a substituted role, a shifted number, or an added cause that was not in the source.

Therefore, model mode without linking entities and numbers to source data and without verifying statements against the source is not used in a news pipeline — flexibility of wording here is bought only by limiting freedom.

The choice of mode is tied to the event type, not to the newsroom’s preference. Tabular occasions — results, quotes, statistics, protocols — go to the template because accuracy, not style, matters there.

Descriptive and overview storylines where context matters go through the model with mandatory confirmation of every fact, a source note, and a stop condition for publication if verification fails.

A mixed case is when the template provides the basis, and the model layer only paraphrases the finished factual material without adding anything of its own.

In support, these are two different disciplines. A template solution lives on a library of constructs and sets of test inputs: acceptance measures the share of correct substitutions and the number of failures on non-standard data.

A model solution lives on a set of reference events with known correct answers, on fact-checking, and on fixing the parameters under which the result is reproduced; any change of model or instruction means repeated acceptance.

Therefore, a project usually uses a hybrid: template as the default mode, model as an extension where wording variability is needed — and a clear boundary between them.

Technical doubts and limitations of the approach

We separate out the limitations: automated journalism is not a “make news” button but a system with clear boundaries. Below are the five questions that newsrooms most often ask.

What to do with incomplete and contradictory data?

Before generation, a verification layer runs: required fields, value ranges, cross-checking of sources. A contradiction is not smoothed over — the material is stopped with an indication of the conflicting fields. A pause is cheaper than plausible but incorrect text.

How are facts separated from assessments and forecasts?

Facts are collected only from confirmed fields and sources, while assessments and forecasts are a separate layer with a mark in the markup. Numbers are verified against the source export, names and titles against the reference dictionary. Anything that fails verification is marked as requiring confirmation.

How does this integrate into the editorial system?

Through standard interfaces: materials enter the newsroom queue as ordinary drafts with fields and marks. The author edits them in the same place as the rest of the output, without a second window or manual transfer. Access rights, edit history, and publication statuses are preserved as is.

Won’t the texts become repetitive?

Repetitiveness is a real limitation of template-based generation, so we build variability into the structure: several presentation schemes, different block order, different wordings of the same fact. For authored genres with a lively intonation, the editor remains an essential link.

Let’s discuss the task: what is needed to start

So that the project start does not stretch into months of correspondence, what you already have is enough at the input.

You do not need to prepare a final technical specification — it is more important to understand what data the material is born from and who is responsible for publication.

It is useful to have at the input:

• List of sources: exports from accounting systems, tables, press releases, statistics — with examples of real records, not just a description of the structure.
• Five to ten examples of current materials with a note on what is assembled manually in them and what you want to hand over to automation.
• Requirements for tone and prohibitions: mandatory wording, unacceptable topics, rules for citing the source and mentioning partners.
• Who makes decisions on the text and within what timeframe reviews the issue: editor, analyst, legal service.
• Desired rhythm of issues: how many materials per hour, whether there are peak windows for events and reporting periods.
• Restrictions on access and personal data: which systems can be read, where approvals are needed.
• Publication points: website, app, newsletter, external platforms — and in what form the finished text is delivered.

This is enough to conduct a short review: look at the client’s data, analyze examples, and record what goes into automation and what remains with the editor. After the review, we return with a timeline estimate, work plan, and composition of the first iteration. Describe the task — we’ll discuss the scope and starting point.