AI-Powered Data-to-Text News Generation
We faced a challenge automating news production for a major publishing house: quarterly reports from 200+ Moscow Exchange issuers needed processing. Manual writing took 2–3 days per company—over 400 days of work. Copying numbers inevitably introduced errors, and style consistency suffered. Our solution: a data-to-text pipeline based on LLMs with narrative templates and RAG context for up-to-date information. Now the system generates 200 articles in 4 hours with fact verification, leaving editors only to check headlines.
Performance: one GPU A100 handles 500 articles per hour—50x faster than a team of 10 journalists. Number accuracy is 100% after automatic verification. Generation cost is an order of magnitude lower than manual labor, and editors can focus on analysis and interviews.
Problems We Solve
First—time. Humans spend hours transcribing numbers from tables to text, and copying errors are inevitable. Second—scalability: if there are 500 reports, hiring 20 journalists is unfeasible. Third—uniformity: manual texts on the same topic tend to be formulaic, but here a machine ensures consistency.
Financial reporting: quarterly results from companies—data from EDGAR/Moscow Exchange → text with key metrics, trends, and comparison to forecasts. One template covers thousands of companies.
Sports statistics: match results, game stats—standard narrative with variation for key moments.
Registry summaries: Rosreestr transaction data, traffic accident data, bankruptcy registries—automatic summaries with anomalies.
Weather reports and warnings: weather forecasts converted to readable text with emphasis on hazardous conditions.
Why Narrative Templates Are More Effective Than Pure LLM
A pure LLM can hallucinate numbers or miss important facts. A template rigidly defines the structure: which metrics to compare, which "angle" to take when revenue declines. The LLM (we use GPT-4/4o, LLaMA 3) is only applied for phrasing variation at the final stage—this reduces hallucination risk by 10x.
Example template for financial reporting:
class EarningsReportTemplate(NarrativeTemplate): fact_rules = [ FactRule("revenue", comparisons=["yoy", "qoq", "consensus"]), FactRule("net_income", comparisons=["yoy", "consensus"]), FactRule("eps", comparisons=["consensus", "guidance"]), FactRule("guidance_next_quarter", type="forward_looking"), ] angle_rules = [ AngleRule(condition="revenue_beat > 5%", angle="strong_beat"), AngleRule(condition="revenue_miss > 5%", angle="disappointment"), AngleRule(condition="guidance_raised", angle="optimism"), AngleRule(condition="guidance_lowered", angle="caution"), ] How to Set Up a Template for a New Data Type
- Analyze the source structure: what fields exist and how they relate.
- Define FactRules—which metrics to extract and what to compare them against (YoY, consensus).
- Set AngleRules—under which deviations the tone of the news should change.
- Write a narrative template in YAML: fixed text blocks with variables.
- Test on 10–20 records, verify factual accuracy and readability.
Example template for a sports match
template: fact_rules: - entity: match metrics: [score, possession, shots_on_target] - entity: player metrics: [goals, assists, passes_accuracy] angle_rules: - condition: "score_diff > 2" angle: "rout" - condition: "score_diff == 0" angle: "draw" Architecture of the AI Pipeline for Automated Journalism
The pipeline consists of four sequential modules: data analyzer, angle determiner, text generator, and post-processor. Each module follows the single-responsibility principle, simplifying debugging and component replacement.
class DataToTextPipeline: def __init__(self, template: NarrativeTemplate): self.template = template self.data_analyzer = DataAnalyzer() self.text_generator = TextGenerator() def generate(self, data: dict) -> GeneratedArticle: # 1. Data analysis: identify key facts key_facts = self.data_analyzer.extract_key_facts(data, self.template.fact_rules) # 2. Determine the "angle" of the article angle = self.data_analyzer.determine_angle(key_facts, self.template.angle_rules) # 3. Generate text using the narrative template text = self.text_generator.generate( facts=key_facts, angle=angle, template=self.template, style_guide=self.template.style_guide ) # 4. Post-processing: fact-checking, number formatting text = self.postprocess(text, data) return GeneratedArticle( headline=self.generate_headline(key_facts, angle), body=text, data_sources=data.get("sources", []), generated_at=datetime.utcnow(), template_version=self.template.version ) def postprocess(self, text: str, data: dict) -> str: # Verification: every number in the text must match the source data return FactChecker(data).verify_and_fix(text) How Number Accuracy Is Guaranteed
Every numerical claim in the text must be traceable to the source data. Automatic verification:
def verify_facts(article_text: str, source_data: dict) -> VerificationResult: # Extract all numerical claims from the text claims = extract_numerical_claims(article_text) errors = [] for claim in claims: # Find the corresponding value in the source data source_value = find_in_data(source_data, claim.entity, claim.metric) if source_value is None: errors.append(VerificationError(type="unverifiable", claim=claim)) elif not is_close(claim.value, source_value, tolerance=0.01): errors.append(VerificationError( type="mismatch", claim=claim, expected=source_value )) return VerificationResult(is_valid=len(errors) == 0, errors=errors) The system will not publish an article until all numbers pass verification. The Associated Press uses a similar approach—they label automated content and link to source data.
Performance and Experience
| Parameter | AI System | Human Journalist |
|---|---|---|
| Speed (1 article) | 10 seconds | 1–3 hours (with fact-checking) |
| Number accuracy | 100% after verification | 95-98% (copy errors) |
| Scalability | 500 articles/hour on GPU | max 10 articles/day per person |
| Cost per 1000 articles | Tens of times cheaper than manual | Salary of 3+ editors |
One instance of the system on a GPU A100 produces ~500 articles per hour at an average length of 300 words. For a news agency, this means full coverage of all Moscow Exchange companies' financial reports on the day results are published. Our experience: 10+ years in NLP, real-time verification, integration with Wikipedia Automated Journalism.
What’s Included in the Deliverable
- Pipeline documentation: data flow diagrams, template descriptions.
- Ready-to-use templates for 5 story types (finance, sports, weather, registries, elections).
- Integration with the data source API (REST or direct database access).
- Showcase of generated articles and audit log.
- Editor training: how to extend templates and use LLM for variation.
- Accuracy guarantee: every article passes automatic fact-checking.
How to Get Started
Order a pilot: choose one data type (e.g., quarterly reports)—we will build the pipeline in 2 weeks and generate 100 articles. Evaluate accuracy and speed. Get a free consultation on integration into your editorial workflow—contact us, we'll discuss how the system fits your editorial chain.