Development of an AI Anomaly Detection System for Telecom Networks

We build AI anomaly detection systems for telecom network operators. Our team has 10+ years of production ML experience and has delivered 40+ projects. A telecom network generates millions of metrics per minute. Traditional static thresholds — like 80% CPU or packet loss > 1% — miss subtle anomalies: slow drifts, correlated degradations across multiple KPIs, atypical traffic patterns. ML-based detection works without fixed thresholds, adapting to the normal behavior of each element. Contact us for a project assessment.

Real Anomalies We Detect

• Volumetric: DDoS attacks, flash crowds, bandwidth saturation.
• Structural: BGP hijacks, route leaks, prefix deaggregation.
• KPI degradation: CPU, memory, interface errors, latency drifts.
• Multivariate: Simultaneous degradation of 5+ KPIs on one element, each individually in range.
• Route instability: Flapping prefixes, BGP session resets.

How We Build the Detection System

We combine multiple models: Prophet for context-dependent thresholds per KPI, Isolation Forest for multivariate anomalies on node metric vectors, and heuristics for traffic and BGP analysis. This covers three layers: univariate time series, multivariate patterns, and network events.

Context-Aware Thresholds with Prophet

Why static thresholds are insufficient:

• Normal router CPU at peak time = 75% (not anomaly)
• CPU at 50% at 3 AM on Saturday = anomaly (possible attack or memory leak)
• Simultaneous 5 KPI degradation on one element = anomaly, even if each is normal alone

import pandas as pd import numpy as np from prophet import Prophet class ContextualAnomalyDetector: def __init__(self, kpi_name: str): self.kpi_name = kpi_name self.prophet_model = Prophet( daily_seasonality=True, weekly_seasonality=True, interval_width=0.99 ) self.fitted = False def fit(self, historical_data: pd.DataFrame): """ historical_data: DataFrame with columns ds (datetime), y (KPI value) Minimum 4 weeks of history for correct seasonality. """ self.prophet_model.fit(historical_data) self.fitted = True def detect(self, current_value: float, current_time: pd.Timestamp) -> dict: future = pd.DataFrame({'ds': [current_time]}) forecast = self.prophet_model.predict(future) yhat = forecast['yhat'].values[0] yhat_lower = forecast['yhat_lower'].values[0] yhat_upper = forecast['yhat_upper'].values[0] is_anomaly = current_value < yhat_lower or current_value > yhat_upper deviation = (current_value - yhat) / (abs(yhat) + 1e-9) return { 'kpi': self.kpi_name, 'value': current_value, 'expected': yhat, 'bounds': (yhat_lower, yhat_upper), 'anomaly': is_anomaly, 'relative_deviation': deviation } 

Why Isolation Forest Fits Telecom

Isolation Forest handles high-dimensional data (up to 100 KPIs per element). It does not require normal distribution and is robust to outliers during training. We train a separate model per network element, adapting to different behaviors of routers, switches, and servers.

from sklearn.ensemble import IsolationForest from sklearn.preprocessing import StandardScaler class NetworkElementAnomalyDetector: """ Each network element has its own Isolation Forest model. Training: 30 days of normal operation. Inference: every 5 minutes on current KPI vector. """ def __init__(self, element_id: str, contamination=0.01): self.element_id = element_id self.scaler = StandardScaler() self.model = IsolationForest( contamination=contamination, n_estimators=100, random_state=42 ) def fit(self, normal_kpi_matrix: np.ndarray): """ normal_kpi_matrix: (N_samples × N_kpis) """ X = self.scaler.fit_transform(normal_kpi_matrix) self.model.fit(X) # Calibrate threshold on normal data scores = self.model.score_samples(X) self.threshold = np.percentile(scores, 1) # 1% false positive rate def score(self, kpi_vector: np.ndarray) -> dict: X = self.scaler.transform([kpi_vector]) raw_score = self.model.score_samples(X)[0] anomaly_score = -raw_score # higher = more anomalous return { 'element_id': self.element_id, 'anomaly_score': float(anomaly_score), 'is_anomaly': raw_score < self.threshold, 'severity': self._score_to_severity(anomaly_score) } def _score_to_severity(self, score): if score > 0.7: return 'critical' if score > 0.5: return 'major' if score > 0.3: return 'minor' return 'normal' 

Traffic Anomaly Detection

How to Distinguish DDoS from Flash Crowd?

We analyze not only volume but also traffic structure: new sources, protocol ratios, peak duration. DDoS usually is homogeneous traffic from one source type; flash crowd is distributed requests from many IPs.

def detect_traffic_anomaly(traffic_matrix: pd.DataFrame, baseline_stats: dict) -> list: """ traffic_matrix: src_ip × dst_ip × bytes per 5 minutes (NetFlow/IPFIX) Traffic anomalies: volumetric (DDoS), structural (BGP hijack), protocol-based """ anomalies = [] # 1. Volumetric anomaly: sudden inbound traffic spike current_total = traffic_matrix['bytes'].sum() baseline_total = baseline_stats['total_bytes_mean'] baseline_std = baseline_stats['total_bytes_std'] volume_z_score = (current_total - baseline_total) / (baseline_std + 1e-9) if volume_z_score > 5: anomalies.append({ 'type': 'volumetric_spike', 'severity': 'critical', 'z_score': volume_z_score, 'possible_cause': 'DDoS attack or flash crowd' }) # 2. New sources: IPs not in baseline current_sources = set(traffic_matrix['src_ip'].unique()) known_sources = baseline_stats.get('known_src_ips', set()) new_sources = current_sources - known_sources if len(new_sources) > baseline_stats.get('new_ip_threshold', 1000): anomalies.append({ 'type': 'new_source_flood', 'severity': 'major', 'new_ips_count': len(new_sources) }) # 3. Protocol anomaly: ICMP or UDP flood protocol_ratios = traffic_matrix.groupby('protocol')['bytes'].sum() / current_total for proto in ['ICMP', 'UDP']: if protocol_ratios.get(proto, 0) > 0.5: anomalies.append({ 'type': f'{proto}_flood', 'severity': 'major', 'ratio': protocol_ratios[proto] }) return anomalies 

BGP and Routing Anomalies

def analyze_bgp_events(bgp_updates: pd.DataFrame, baseline_prefix_count: int) -> dict: """ BGP hijack: sudden appearance of a new AS path for a known prefix. BGP leak: routes from one provider advertised to another. Route flap: frequent updates = connectivity instability. """ # Route flapping prefix_update_counts = bgp_updates.groupby('prefix').size() flapping_prefixes = prefix_update_counts[prefix_update_counts > 10].index.tolist() # New AS-origin for known prefixes known_origins = {} # prefix → expected AS hijack_candidates = [] for _, row in bgp_updates.iterrows(): if row['prefix'] in known_origins: if row['origin_as'] != known_origins[row['prefix']]: hijack_candidates.append({ 'prefix': row['prefix'], 'expected_as': known_origins[row['prefix']], 'detected_as': row['origin_as'] }) return { 'flapping_prefixes': flapping_prefixes, 'hijack_candidates': hijack_candidates, 'route_instability': len(flapping_prefixes) > 5 } 

Alert Correlation and Noise Suppression

When a router uplink fails, hundreds of downstream anomalies appear. Our algorithm: build a dependency graph from CMDB topology → identify upstream source → group into one incident.

Timeline: Prophet + Isolation Forest + Traffic anomaly — 3-4 weeks. BGP anomaly, alert correlation graph, automatic RCA, NOC integration — 2-3 months.

Performance Metrics

On a typical telecom network with 500–5000 elements, the system achieves:

• Precision of KPI anomaly detection: ≥90% (Isolation Forest, 30-minute sliding window).
• Recall for critical incidents (outage, DDoS): ≥95%.
• MTTD (mean time to detect): reduced from 15–30 minutes with manual monitoring to 2–5 minutes.
• Noise reduction: 70–80% fewer alerts due to correlation and dedup.
• Inference latency: Prophet — 50 ms per element, Isolation Forest — 5 ms per vector.

Case study: In a deployment for a major telecom provider, we cut MTTD from 8 minutes to 1.2 minutes and reduced false positives by 80%, enabling the NOC to handle 5x more incidents without adding staff.

Monitoring covers three data sources: SNMP/gRPC streams from nodes, NetFlow/IPFIX traffic data, BGP MRT dumps and syslog events. Each layer is detected independently, then events merge in the correlation engine. The system supports incremental retraining once a week without service interruption.

What's Included in the Work

Static Thresholds vs ML: A Comparison

ML detection finds 3x more anomalies than static thresholds and reduces noise by 5x.

How to Get Started

Contact us for a preliminary assessment of your network. We'll analyze available data, identify typical anomalies, and propose an architecture. Get a free consultation.