A developer trained a segmentation model in Keras, but on hardware it froze. The model didn't fit into 2 MB of STM32 Flash, and FP32 weights dropped from 4 MB to 1.2 MB after quantization to TFLite Micro, but accuracy fell by 12%—and the client lost the order. We know how to avoid such scenarios: over six years we've done dozens of conversions for MCU, Arm Linux, and Google Coral. End-to-end: model analysis, format selection, quantization, accuracy validation, deployment. Our track record: 50+ edge projects, 30+ for Coral. Contact us to assess your project in one day. Conversion cost is calculated individually, but on average the project pays for itself through reduced compute costs and faster inference.
Problems We Solve
Operation Incompatibility
TFLite Micro supports only a subset of full TensorFlow: ~250 operations vs. ~2000. Common ops like tf.nn.depthwise_conv2d, tf.reshape are present, but tf.where or tf.sort are missing. We manually replace unsupported layers with equivalents—for example, replacing tf.where with tf.cast combined with tf.multiply. This issue is especially acute for edge ML, where every operation counts.
Model Size and Quantization
Edge TPU only accepts INT8 models, and they must be 8 MB or less. Our team has experience adapting YOLOv5 (14 MB float) to 4.2 MB INT8 with mAP drop of no more than 2%. We use quantization-aware training to preserve accuracy. Compared to Float16, INT8 quantization delivers 3–4x higher speed on Edge TPU at the same energy cost. TFLite Micro is 50% more compact than standard TFLite, which is critical for MCUs.
Performance Drop on MCU
Even after conversion to TFLite Micro, a model may be slow due to suboptimal operation ordering. We profile each operation and modify the graph to reduce DMA calls—gaining up to 40% on STM32H7. This is especially important for ML on STM32, where resources are tight.
How We Do It
Conversion pipeline for each platform.
TFLite (Mobile / Raspberry Pi / x86 Edge)
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir) converter.optimizations = [tf.lite.Optimize.DEFAULT] tflite_model = converter.convert() Supports: INT8, FP16, dynamic range quantization. GPU delegate, NNAPI, Hexagon DSP. Ideal for ML on Raspberry Pi.
TFLite Micro (MCU, <1 MB)
Subset of TFLite operations, portable C++:
xxd -i model.tflite > model_data.cc # convert to C array Supported on: STM32, Arduino, ESP32, nRF52840. Compatibility checker is mandatory—we run it before starting work.
Edge TPU (Google Coral)
Edge TPU requires INT8 quantization. Only operations from the whitelist execute on hardware (rest is CPU fallback):
edgetpu_compiler model_quant.tflite # Google Coral compiler Performance: 4 TOPS (Coral USB), 4 TOPS (Coral PCIe M.2). Great for image classification and object detection. As per Google Coral documentation, keep the model <8 MB for full acceleration.
Platform Comparison
| Platform | Devices | Max Model Size | Quantization | Tools |
|---|---|---|---|---|
| TFLite | Android, iOS, RPi, x86 | No limit | FP16, INT8, dynamic | TFLite Converter, GPU Delegate |
| TFLite Micro | STM32, ESP32, Arduino | <1 MB Flash | INT8 mandatory | XXD, compatibility checker |
| Edge TPU | Coral USB/PCIe/M.2 | 8 MB (full acceleration) | INT8 mandatory | edgetpu_compiler |
Quantization Types and Parameters
| Type | Weight Size | Accuracy Loss | Hardware Acceleration |
|---|---|---|---|
| FP32 | 4 bytes | Baseline | CPU/GPU |
| FP16 | 2 bytes | <1% | GPU, some TPUs |
| Dynamic range | 2–4 bytes | 1–3% | CPU (optimization) |
| INT8 | 1 byte | 1–5% | Edge TPU, DSP, MCU |
Why INT8 Quantization Is the Standard for Edge TPU?
Edge TPU hardware operates on integers—float operations are emulated on CPU with a 10–20x speed drop. We use calibration on a representative dataset to find scales and zero points. For image models, mAP loss is typically 1–3%.
How to Check Model Compatibility with TFLite Micro?
We run tflite_micro_compatibility_checker even before conversion. If an unsupported operation is found, we replace it with an equivalent. For example, tf.nn.max_pool can be replaced with tf.nn.avg_pool if the task allows. As a last resort, we use a custom operator, but that complicates deployment.
Detailed compatibility check workflow
- Load model in .tflite format.
- Run through checker: get list of unsupported operations.
- For each operation, find a replacement from the available set.
- Re-run compatibility check.
- If replacement is impossible, consider custom operator or platform change.
Process of Work
- Model analysis: load, profile operations, estimate size.
- Platform selection: MCU, SBC, or Edge TPU—pick the optimal option.
- Conversion and quantization: apply QAT or post-training quantization.
- Accuracy validation: compare float and quantized model outputs on test set.
- Deployment: prepare C-array, test on target device.
What's Included
- Documentation: conversion report, deployment instructions.
- Source code of conversion and validation scripts.
- Training for the client's team (1–2 sessions).
- Accuracy guarantee: deviation no more than 5% from baseline.
- Post-deployment support for 1 month.
Timeline and Budget
Timelines: from 1 to 3 weeks depending on model complexity and requirements. Cost is calculated individually—contact us to assess your project within one business day. Get a consultation and a commercial proposal tailored to your needs. Our experience: over 6 years in edge ML, 50+ projects, 30+ for Coral. Savings at the deployment stage are one of the key results of our projects.