A model does not fit into a microcontroller: where a TinyML project breaks
A trained model works reliably on a server, but after porting the trained model to a microcontroller it starts producing garbage or does not even build — this is the most common failure point.
The reason is not the quality of training, but that the microcontroller's constraints are set by hardware: RAM is measured in hundreds of kilobytes, weights and code share it, and hardware support for floating-point operations is often absent.
The model does not fit into the device memory — and instead of running model inference on the device, rewriting the architecture begins.
The second fault line is bit width. Weights in 32-bit format take four times more space than needed, while integer arithmetic runs faster on such chips.
Quantization to 8 bits reduces size severalfold, but requires calibration on real data: without it, accuracy drops specifically on the rare classes for which the project was undertaken.
The third is the runtime environment. A layer that lives comfortably in the training environment may simply be absent on the device: the required operation is missing, dynamic tensor shapes are unsupported, a specific activation layer is unavailable.
Then the graph is rebuilt from the set of operators that the device can actually execute, and the output divergence from the original model is checked.
TinyML on a microcontroller breaks not in training, but at the intersection of architecture, weight format, and runtime environment. Below we examine how this is addressed in practice.
What changes when model inference moves onto the device itself
When the model runs on the device itself, processing no longer depends on the communication channel: the sensor captures the signal itself, computes the result, and exposes only the inference output.
Response latency stops "floating" — it is determined by the local polling cycle and remains the same under network congestion, weak signal, and when the gateway is unavailable.
Autonomous operation of the sensor is a consequence of the same decision. The device continues measuring and classifying while there is no connection, and a compressed summary goes to the network after it is restored.
The volume of transmitted data drops severalfold: instead of a continuous stream of raw samples, events and interval aggregates are transmitted, and the channel stops being a bottleneck.
Fault tolerance without a network stops being a separate task.
A whole class of failures on the transmission channel side goes away — disconnections, packet loss, receiver buffer overflow — because the critical part of the computation is performed locally and can no longer be lost in transit.
We verify the effect measurably: what share of model inferences is computed on the device, how long the device operates autonomously without a connection, by what factor traffic decreased, how the response behaves under channel degradation, and whether accumulated data is restored after reconnecting to the network.
It is precisely on these metrics that the TinyML decision is built — from model selection to packaging it for the memory and energy budget of a specific device.
TinyML, classic algorithms, or cloud: what to choose
The choice between TinyML and cloud is determined not by fashion, but by three microcontroller constraints: RAM size, energy budget for autonomous power, and allowable response latency. First we calculate these boundaries, and only then decide where the model can live at all.
The second question is what exactly solves the task. Threshold algorithms on features are cheaper and more predictable than any neural network, but they break down when classes overlap.
If a trainable model cannot be avoided, it is compressed — weight quantization, pruning of insignificant connections, architecture simplification — and accuracy is measured again on your own data, not according to someone else's benchmarks.
| Approach | When it fits | Limitations |
|---|---|---|
| Threshold algorithms on the device | The feature is well described by a rule: vibration threshold, amplitude, event counter. Predictability and minimal consumption are needed | Does not work with overlapping classes and signal drift; each new condition means manual tuning |
| Compressed model on the device (TinyML) | Offline pattern and anomaly recognition from sensors, response is time-critical | Hard ceiling on model size and accuracy; updating the model means reflashing the fleet of devices |
| Cloud inference | Maximum accuracy is needed, models are updated frequently, connectivity and power are stable | Channel latency, dependence on connectivity, transmission of raw data, and privacy requirements |
| Hybrid inference scheme | Coarse event filtering on the device, only selected data goes to the cloud | Harder to debug: two model versions, synchronization of logic and connection interruptions |
The choice is fixed by measurement on a pilot batch: miss rate, number of false positives, response time, daily consumption. The hybrid more often wins where high accuracy is needed and the channel breaks — but maintenance is more expensive.
Stages of developing a model for a microcontroller
In TinyML, the device's memory and power constraints are set at the beginning of the project, not checked at the end. Therefore, the TinyML development stages go from task audit to project handover, and each step is closed with validation on a real controller.
- Task and data audit. We look at which quantities are captured from sensors, at what frequency, and what has already been accumulated in logs. We fix acceptance metrics, allowable response time, and the amount of memory available to the model on the device.
- Solution design and sensor data collection. We determine the composition of channels and labeling, then collect data on real product instances, not only on a prototype, so that noise, drift, and installation specifics make it into the sample.
- Feature preparation and prototype. We compute features, choose an architecture for the available memory, and train the model with an accuracy margin — it will have to withstand compression.
- Model training and compression. We prune weights, convert computations to integer form, and apply distillation if necessary. After each step we compare accuracy with the original: critical signal classes must not degrade.
- Validation on the device. We run the model on the target controller, measure response time and consumption, and check behavior on boundary and noisy signals.
- Project handover. We deliver a reproducible training process, a dataset with labeling description, firmware, and metric reports. We train your team to fine-tune the model on new data and release updates independently.
Project artifacts: model, firmware, documentation
A TinyML project is considered delivered not when the model has shown the required accuracy on our bench, but when it operates stably on the customer's device and can be maintained without us.
Therefore the package includes everything needed for rebuild, validation, and update — from training scripts to procedures.
- Training source code: dataset preparation, training, quantization, and conversion to the format accepted by the microcontroller compiler.
- Microcontroller firmware: a ready build for a specific board — with assigned pins, sensor polling frequency, and power-saving mode.
- Memory layout map: how much is occupied by weights, intermediate tensors, and buffers, and what reserve remains for application logic and future edits.
- Test set and reference outputs: running on it shows that the model has not lost accuracy when transferred from server to device.
- Model documentation: purpose, input and output signals, allowable ranges, conditions under which quality drops, expected metrics.
- Update procedure: how to retrain on new data, tag a version, roll it out to the fleet of devices, and revert to the previous firmware.
- Build and flashing scheme: dependencies, commands, and step order that your engineer reproduces locally without our help.
Separately, we discuss what of this is critical for your fleet: where the reference set is stored, who owns the right to retrain, and how model changes are recorded between releases.
Such a package removes dependence on a single contractor — maintenance and evolution of the solution remain within your team.
Practice: a model for a vibration sensor on TinyML
Before the project, vibration sensors sent a raw stream of measurements to the server: the channel was occupied almost continuously, and the decision to stop was made by a person according to the inspection schedule.
The growth of amplitude at characteristic frequencies was not always noticed in time — bearing assemblies failed between scheduled checks.
What was changed
After implementing TinyML, the engineering part looked like this: recordings of normal and fault modes were collected, labeled by wear stages, and a compact model was trained that computes features directly on the microcontroller.
Inference runs on TensorFlow Lite for Microcontrollers, weights are quantized, and the model together with the sample buffer occupies a small fraction of memory — external memory and hardware replacement were not required.
Metrics before and after
The volume of transmitted data dropped by two orders of magnitude: events with a window of samples go upward, not a continuous stream, and the channel is free for service traffic.
Model accuracy on the device is 0.94 with a false alarm rate below 3%, so equipment failure prediction triggers at an early wear stage, when there is time for planned replacement of the assembly.
The remaining misses relate to slowly developing failures — the model does not have time to see the dynamics in a short window, and this is addressed by increasing the analysis window, not by growing the model itself. Firmware updates with new weights are delivered over the air, and the fleet is migrated in stages.
What to do if a trained model does not fit into the device?
In TinyML projects, a trained model does not fit into the device — this is not a dead end, but the point where engineering work begins. First we calculate the memory budget: how much is occupied by weights, and how much by intermediate tensors and working buffers for inference.
It often turns out that the weights fit, but there is not enough memory for activations — and then the solution is one thing, not another.
The first step is architecture simplification: we reduce the number of channels and layers, replace heavy convolutions with separable ones, and remove branches that contribute fractions of a percent of quality.
This is model compression without losing the meaning of the task: accuracy drops by a few percent, while size falls severalfold.
The second step is quantizing weights into integer representation. We calibrate parameters on real sensor data, not synthetic data: otherwise the discrepancy between bench and field will eat up all the savings.
The model becomes smaller and faster, while the original version is preserved — rollback is possible at any stage.
If this is still not enough — retraining for the device. We change the input, split the task into a cascade of simple models, and transfer knowledge from a large network into a compact one.
Training is done with the target hardware in mind: we check not only the quality metric, but also inference time with peak memory.
And only at the end — choosing a different class of microcontroller: more memory, a hardware accelerator, or an external module. This is the longest path — changing the batch, recalibration, field checks — so we keep it as the last option, not the first.
Engineering team doubts: we answer to the point
TinyML development is usually slowed down not by models, but by questions the team did not get a clear answer to before starting. Below are the ones heard most often during review: they determine whether the solution reaches production or remains on the bench.
How much will accuracy drop after compression?
Quantization and pruning almost always remove part of the metric — we plan for a controlled tolerance, not zero loss. We fix the tolerance separately for each significant class: a rare class drops more than average accuracy, and it is precisely that which determines suitability.
How to update the model on already deployed devices?
We provide two channels: full firmware replacement and weight-only update when the model is decoupled from the logic — the second is cheaper in traffic and risk.
Two memory partitions for versions, integrity verification, and automatic rollback if self-check after update fails are mandatory.
There is little training data — where to start?
We go through transfer from the cloud: train a large model on available data and transfer knowledge to a compact one, supplementing with synthetic data and signal augmentation.
We gather the remaining labels actively — the device sends only ambiguous detections, and we label them selectively.
How to notice model degradation in the field?
By telemetry: model confidence, share of decision abstentions, and divergence of the input signal distribution from the training sample.
When a threshold is crossed, the device accumulates data, and a bench with a reference sample runs regression — a shift in conditions is visible before the error begins to grow.
Let's discuss the task and return with an estimate and plan
To estimate a TinyML project, a ready technical specification is not needed — a description of the task and operating conditions is enough.
We review the inputs, clarify the device's memory and power consumption constraints, and return with a work plan: stages, timelines, and what will be required from your side.
For the initial review, it is useful to send the input data:
- task description: what the device should recognize or predict and how often — continuously or on event;
- data: sensor recordings, labeled fragments, or raw logs from a real device;
- platform: microcontroller, memory size, and allowable power consumption, if already selected;
- constraints: response latency, autonomy, requirements for updating the model on the device;
- what has already been done: prototypes, trained models, accuracy measurements;
- acceptance criteria: by which metric you will understand that the solution works, and on which data to verify it;
- timelines: whether there is a date by which a working prototype is needed.
The discussion is a single 40–60 minute conversation with the engineer who will lead the project, without retelling through intermediaries.
After it, we return with a written work plan: composition of stages, model design and its constraints, requirements for data and labeling, procedure for validation on the device, and a list of risks. Describe the task — we will return with an estimate and plan.
