AI IoT Data Forecasting
We often see the same pain: sensors on the factory floor are noisy, data gaps appear, and standard SARIMA gives a 30% error because it ignores interactions with neighboring equipment. The client wants to predict a compressor failure an hour ahead, but the current model is either too heavy for edge or misses anomalies. We build turnkey systems that close these gaps. Over 5 years, we have deployed predictive pipelines on 50+ industrial sites, processing over 10,000 sensors in real time. Typical project budget ranges from $25k to $60k, with recurring savings of $10k/year per sensor on cloud costs.
Why Standard Models Fall Short for IoT Forecasting
IoT time series have specific characteristics: sampling rates in seconds/minutes, sensor noise, outliers (5σ and above), long gaps due to channel failures, and drift. Univariate models ignore dependencies — workshop temperature depends on weather, machine load, and open doors. Multivariate forecasting requires feature engineering: lag features, rolling statistics, time stamps.
Data quality issues are the main reason 60% of proofs-of-concept fail. We assess:
def assess_data_quality(ts_df): issues = {} issues['missing_rate'] = ts_df.isna().mean() z_scores = (ts_df - ts_df.mean()) / ts_df.std() issues['outlier_rate'] = (np.abs(z_scores) > 5).mean() issues['stuck_periods'] = detect_constant_windows(ts_df, min_duration=10) long_trend = np.polyfit(range(len(ts_df)), ts_df.fillna(method='ffill'), 1)[0] issues['drift_per_day'] = long_trend * 86400 / ts_df.index.freq.nanos * 1e9 return issues How We Build the Preprocessing Pipeline
- Interpolation of short gaps (<5 min) — time-linear.
- Forward fill for long gaps with an imputation flag (model learns to ignore fictitious values).
- Adaptive normalization: a 24-hour rolling window compensates drift and seasonality.
- Feature synthesis: lags [1, 5, 15, 60, 1440], rolling mean/std 5/15/60 min, hour/day/day_of_week.
LightGBM with these features achieves 15–20% higher accuracy than SARIMA on regular series and 2 times better on noisy ones.
Model Selection Based on Series Characteristics
| Model | Best Suited For | Accuracy* | Inference Time | Edge-ready |
|---|---|---|---|---|
| SARIMA | Series with seasonality (energy, temperature) | 85–92% MAPE | 50 µs | Yes (C++) |
| LightGBM | Any: noisy, multivariate | 90–95% | 100 µs | Yes (C API) |
| LSTM | Nonlinear patterns, long dependencies | 92–97% | 1–5 ms | Yes (INT8 ONNX) |
| Chronos/TimesFM | Rare sensors, zero-shot | 75–85% | 10–50 ms | No (cloud) |
*MAPE at 12-step (minute) forecast horizon. LightGBM is 2 times more accurate than SARIMA on noisy data, and LSTM achieves 97% MAPE vs 85% for SARIMA on complex series.
Typical Problems Solved by Preprocessing
| Problem | Method | Impact on Accuracy |
|---|---|---|
| Gaps <5 min | Linear interpolation | ±2% |
| Gaps >5 min | Forward fill + flag | ±5% |
| Sensor drift | Rolling normalization (24h window) | +10–15% |
| Outliers (5σ+) | Percentile capping | +5–8% |
How to Deploy the Model on Edge
Edge devices — ARM Cortex-A, 256 KB RAM. Our solution:
- Train the model (PyTorch).
- Export to ONNX.
- Quantize to INT8 (4x size reduction, 2x speedup).
- Run on device via ONNX Runtime.
import onnxruntime as ort torch.onnx.export(model, dummy_input, 'forecaster_edge.onnx', opset_version=11) from onnxruntime.quantization import quantize_dynamic quantize_dynamic('forecaster_edge.onnx', 'forecaster_edge_quant.onnx') session = ort.InferenceSession('forecaster_edge_quant.onnx') forecast = session.run(None, {'input': recent_data})[0] Edge forecasting saves traffic: only anomalies are sent to the cloud — 80–95% savings, making edge inference 5 to 20 times more efficient in data transmission. This is especially critical for remote sites with limited bandwidth. We use ONNX quantization to reduce model size and leverage TinyML techniques for efficient on-device inference. Our approach combines machine learning for IoT time series forecasting. We design an MQTT Kafka pipeline for sensor data. Our solution includes Grafana monitoring dashboards. Contact us for a one-week pilot to test the effect on your data.
What We Do for 10,000+ Sensors
- Hierarchical forecasting: grouping by type, location, system → top-down reconciliation improves stability.
- AutoML time series: StatsForecast parallel-fit SARIMA/ETS for thousands of series in minutes.
- Online sensor anomaly detection: rolling z-score + residual-based (forecast-actual).
Case study: for a network of pump stations, we deployed a hierarchical LightGBM that reduced false positive anomaly alerts by 30% compared to unitary models.
How the Process Works
- Analytics: audit sources (MQTT/Kafka), assess quality, select model.
- Design: pipeline architecture, data schema, tech stack (MLflow, Weights & Biases).
- Implementation: preprocessing pipeline → baseline LightGBM → iterations (LSTM, edge).
- Testing: A/B on historical data, p99 latency, reliability.
- Deployment: Docker + Kubernetes (cloud) or ONNX (edge) + Grafana dashboard.
What’s Included in the Deliverable
- Collection and preprocessing pipeline (Python, Kafka, MQTT).
- Trained model (online + batch).
- Grafana dashboard with forecasts and alerting.
- Edge inference code (ONNX / C++).
- Documentation (architecture, API, instructions).
- Operator training (2–3 hours).
- API documentation.
- Access to cloud dashboard.
- 3 months of post-deployment support.
Timeline
- 4–5 weeks: ingestion + preprocessing + LightGBM baseline + dashboard.
- 3–4 months: full cycle with LSTM, edge ONNX, streaming anomaly detection, AutoML for fleet.
We provide a free project assessment: reach out to discuss your use case. Get a consultation on model selection. ISO 27001 certified, 5+ years of experience — quality guaranteed.
Additional resources: Wikipedia - Time series.