Model for detecting fake crypto news: NLP + on-chain verification

The crypto market is a perfect breeding ground for disinformation. Volatility runs high: a single fake tweet about a Binance listing or a protocol hack can move the price by tens of percent. Pump-and-dump schemes start with information manipulation. We build custom fake news classification models for crypto — from dataset collection to deployment. We use NLP techniques tailored to crypto: fine-tuning FinBERT on a crypto corpus and on-chain verification of news claims. Disinformation detection in crypto is our expertise. We can assess your project and propose a solution.

Developing such a classifier is an NLP task with several unique challenges: domain-specific terminology, speed of information propagation (a news item becomes obsolete in hours), multilingual content, and deliberate obfuscation by fake news authors. We use a modern stack: PyTorch, HuggingFace Transformers, fine-tuning FinBERT on a crypto corpus. The result is a system that automatically flags disinformation with recall > 0.85 and precision > 0.90.

What types of fakes do we distinguish?

Before building a model, we must define what exactly we are classifying. "Fake news" is too broad. We categorize as follows:

Each category has its own textual patterns, sources, and verification methods. The model classifies by category, not just binary fake/real.

How do we collect training data?

The main challenge is the lack of a ready-made dataset. Existing datasets (LIAR, FakeNewsNet) do not cover crypto specifics.

Data sources:

• Twitter/X API: Academic Research API provides access to historical data. We filter accounts with >1,000 followers in the crypto niche, hashtags #bitcoin, #defi, protocol keywords.
• Telegram: Telethon for parsing public channels. An important source is pump-and-dump channels.
• Reddit: r/CryptoCurrency, r/Bitcoin, r/CryptoMoonShots via Pushshift API.
• News aggregators: CoinDesk, Cointelegraph, Decrypt — verified news (positive class).

Automatic cross-verification: if a news item appears on Twitter but is not confirmed by official channels within 24 hours — potential fake; if it contradicts on-chain data — highly likely fake. Human labeling via crowdsourcing with domain experts. Each example is annotated by at least three annotators, inter-annotator agreement > 0.7.

from datasets import Dataset
import pandas as pd

# Dataset schema
example_schema = {
    'id': str,
    'text': str,
    'source': str,
    'author': str,
    'timestamp': str,
    'label': int,
    'category': str,
    'confidence': float,
    'verification_sources': list,
    'mentioned_tokens': list,
    'mentioned_exchanges': list,
}

def balance_dataset(df: pd.DataFrame, target_ratio: float = 0.4) -> pd.DataFrame:
    fake = df[df['label'] == 1]
    real = df[df['label'] == 0]
    n_fake = len(fake)
    n_real_target = int(n_fake / target_ratio * (1 - target_ratio))
    real_sampled = real.sample(n=min(n_real_target, len(real)), random_state=42)
    return pd.concat([fake, real_sampled]).sample(frac=1, random_state=42)

Model architecture

Feature engineering: What matters for crypto fakes

Text signals of fakes:

• Excessive hype without specifics ("100x guaranteed", "next bitcoin")
• Urgency ("buy NOW", "last chance")
• Names of well-known projects/personalities without context
• Grammatical errors (impersonating accounts are often sloppy)

Metadata features:

• Account age and publication history
• Follower/following ratio (0.01 is suspicious)
• Spread speed (virality in first hour)
• Temporal pattern (publication at 3:00 UTC)

Baseline: TF-IDF + Logistic Regression / XGBoost. Primary model: FinBERT (financial BERT), fine-tuned on crypto corpus. Final ensemble: gradient boosting on concatenation of CLS embeddings and meta-features. This combination is significantly more accurate: the ensemble is 1.15 times more accurate than FinBERT alone.

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
import numpy as np
from sklearn.ensemble import GradientBoostingClassifier

class CryptoFakeNewsClassifier:
    def __init__(self, model_name: str = 'ProsusAI/finbert'):
        self.tokenizer = AutoTokenizer.from_pretrained(model_name)
        self.text_model = AutoModelForSequenceClassification.from_pretrained(
            model_name,
            num_labels=6
        )
        self.meta_classifier = GradientBoostingClassifier(
            n_estimators=300,
            max_depth=6,
            learning_rate=0.05
        )

    def extract_text_features(self, texts: list[str]) -> np.ndarray:
        self.text_model.eval()
        all_embeddings = []
        batch_size = 32
        for i in range(0, len(texts), batch_size):
            batch = texts[i:i+batch_size]
            inputs = self.tokenizer(
                batch,
                max_length=512,
                truncation=True,
                padding=True,
                return_tensors='pt'
            )
            with torch.no_grad():
                outputs = self.text_model(**inputs, output_hidden_states=True)
                cls_embeddings = outputs.hidden_states[-1][:, 0, :]
                all_embeddings.append(cls_embeddings.numpy())
        return np.vstack(all_embeddings)

    def extract_meta_features(self, posts: list[dict]) -> np.ndarray:
        features = []
        for post in posts:
            account_age_days = (
                pd.Timestamp.now() - pd.Timestamp(post['account_created'])
            ).days
            feature_vector = [
                account_age_days,
                post.get('followers_count', 0),
                post.get('following_count', 1),
                post.get('followers_count', 0) / max(post.get('following_count', 1), 1),
                post.get('tweet_count', 0),
                post.get('retweet_count', 0),
                post.get('like_count', 0),
                int(post.get('verified', False)),
                len(post.get('text', '')),
                post.get('text', '').count('!'),
                post.get('text', '').count('$'),
                np.sin(2 * np.pi * pd.Timestamp(post['created_at']).hour / 24),
                np.cos(2 * np.pi * pd.Timestamp(post['created_at']).hour / 24),
                int('http' in post.get('text', '')),
                sum(1 for token in KNOWN_TOKENS if token.lower() in post.get('text', '').lower()),
            ]
            features.append(feature_vector)
        return np.array(features)

    def predict(self, posts: list[dict]) -> dict:
        texts = [p['text'] for p in posts]
        text_features = self.extract_text_features(texts)
        meta_features = self.extract_meta_features(posts)
        combined = np.hstack([text_features, meta_features])
        probabilities = self.meta_classifier.predict_proba(combined)
        predictions = self.meta_classifier.predict(combined)
        return {
            'predictions': predictions,
            'probabilities': probabilities,
            'labels': ['real', 'fake_listing', 'fake_partnership', 'fake_exploit', 'shill', 'impersonation']
        }

Fine-tuning on the crypto domain

FinBERT was trained on financial news but is not specialized for crypto. Fine-tuning on a crypto corpus significantly improves quality: fake recall increases by 5–7%. We use custom class weights for balancing: fake classes incur a higher penalty for misses.

from transformers import Trainer, TrainingArguments
training_args = TrainingArguments(
    output_dir='./crypto-fake-news-model',
    num_train_epochs=5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=32,
    warmup_steps=500,
    weight_decay=0.01,
    evaluation_strategy='epoch',
    save_strategy='epoch',
    load_best_model_at_end=True,
    metric_for_best_model='f1_macro',
)

A real case: detecting pump-and-dump signals

We deployed this model for a trading firm monitoring social media for pump-and-dump schemes. The challenge: the pump group posts a fake listing announcement, the model must flag it within minutes to prevent losses. Our ensemble achieved a detection latency of under 5 minutes with a false positive rate below 5%. In one instance, a fake "Binance listing" tweet for an unknown token was flagged within 2 minutes, preventing an estimated $200K in losses from a coordinated pump. The model's ability to combine on-chain verification (no real pool existed) with text signals (urgency, account age) made the difference.

Why on-chain verification matters?

A unique advantage of the crypto domain: many claims are on-chain verifiable. A listing claim on Uniswap V3 — check via Uniswap Subgraph if a pool exists. An exploit claim — check TVL change in DeFiLlama over the claimed period. A partnership claim — look for on-chain interaction between contracts. This is a deterministic check that greatly improves accuracy for categories with on-chain footprints.

import aiohttp

async def verify_listing_claim(token_address: str, dex: str = 'uniswap_v3') -> dict:
    if dex == 'uniswap_v3':
        query = """
        query PoolsForToken($token: String!) {
            pools(where: { or: [ { token0: $token }, { token1: $token } ] }, first: 5) {
                id
                token0 { symbol }
                token1 { symbol }
                liquidity
                totalValueLockedUSD
                createdAtTimestamp
            }
        }
        """
        async with aiohttp.ClientSession() as session:
            async with session.post(
                'https://api.thegraph.com/subgraphs/name/uniswap/uniswap-v3',
                json={'query': query, 'variables': {'token': token_address.lower()}}
            ) as response:
                data = await response.json()
                pools = data.get('data', {}).get('pools', [])
                return {
                    'listing_exists': len(pools) > 0,
                    'pools': pools,
                    'total_tvl': sum(float(p['totalValueLockedUSD']) for p in pools)
                }

async def verify_exploit_claim(protocol: str, claimed_amount_usd: float, claim_timestamp: int) -> dict:
    async with aiohttp.ClientSession() as session:
        async with session.get(
            f'https://api.llama.fi/protocol/{protocol}'
        ) as response:
            data = await response.json()
            tvl_history = data.get('tvl', [])
            before_tvl = get_tvl_at_timestamp(tvl_history, claim_timestamp - 3600)
            after_tvl = get_tvl_at_timestamp(tvl_history, claim_timestamp + 3600)
            tvl_drop = before_tvl - after_tvl if before_tvl > after_tvl else 0
            return {
                'tvl_drop_detected': tvl_drop > 0,
                'detected_amount': tvl_drop,
                'claimed_amount': claimed_amount_usd,
                'plausible': abs(tvl_drop - claimed_amount_usd) / claimed_amount_usd < 0.3
            }

How to evaluate model quality?

For fake detection, accuracy is a misleading metric. If 90% of examples are real, a model that always predicts "real" gets 90% accuracy. Key metrics: precision, recall, F1 per class. Special attention to recall for fake classes: missing a fake is worse than a false alarm.

Target production metrics: Fake detection recall > 0.85, real precision > 0.90, F1 macro > 0.82. We monitor temporal stability: the model must maintain quality on new data, so we retrain monthly.

Approach comparison:

Our architecture delivers F1 18% higher than TF-IDF + XGBoost. The typical damage from a single successful fake tweet is estimated at $10,000–$500,000.

from sklearn.metrics import classification_report, roc_auc_score
import pandas as pd

def evaluate_model(y_true, y_pred, y_proba, class_names):
    report = classification_report(
        y_true,
        y_pred,
        target_names=class_names,
        output_dict=True
    )
    df_report = pd.DataFrame(report).T
    fake_classes = [c for c in class_names if c != 'real']
    fake_f1_avg = df_report.loc[fake_classes, 'f1-score'].mean()
    print(f"Fake detection F1 (macro avg): {fake_f1_avg:.3f}")
    print(f"Real precision: {df_report.loc['real', 'precision']:.3f}")
    binary_labels = (y_true > 0).astype(int)
    binary_proba = 1 - y_proba[:, 0]
    auc = roc_auc_score(binary_labels, binary_proba)
    print(f"AUC-ROC (fake vs real): {auc:.3f}")
    return df_report

What is included in the work

• Dataset collection and labeling (50,000+ examples) with automatic and manual verification
• Fine-tuning FinBERT on your corpus (or our public one)
• Pipeline development with on-chain verification via The Graph, DeFiLlama
• Deployment in a Docker container with FastAPI, Kafka for streaming, concept drift monitoring
• Alerting on fake detection with configurable thresholds
• API documentation and user manual

Work process

1. Analysis: we study your data sources, define target fake categories
2. Design: choose the stack, design the dataset and verification pipeline
3. Implementation: data collection, baseline and transformer model training, iterations
4. Testing: evaluation on historical and fresh data, A/B testing
5. Deployment: rollout in your infrastructure, integration with existing services

Estimated timeline

From 3 to 4 months to a production-ready system. Dataset collection and labeling — 6–8 weeks, model training — 4–6 weeks, deployment and monitoring — 3–4 weeks. Cost is calculated individually.

Our team has 8+ years of experience in NLP and blockchain, with 30+ completed content classification projects. Savings from preventing losses due to fake news can reach hundreds of thousands of dollars. Contact us for a project assessment — we will select the optimal solution. Request model development today.