IoT logging in a mobile app: where events get lost
IoT logging in a mobile app breaks not on the server, but on the client side — at the moment when a device event is supposed to turn into a record.
A typical failure: the sensor fired, the app generated an event, but connectivity dropped for a few seconds, the request went nowhere, and a gap appeared in the telemetry.
The second scenario — the user minimized the app, the system suspended background tasks, and the accumulated events never left the device.
The third class of problems is duplicates. Re-sending after a timeout without an event identifier causes the same record to arrive at the server twice, and per-device analytics starts lying.
The fourth is log growth: if every event is written to local storage without rotation, the database grows, and the UI starts to lag when opening the device list.
All four failures are cured by a single architecture: an on-device offline queue with delivery guarantees, idempotent event identifiers, and background telemetry synchronization that survives app minimization. Let's break down how this is structured and what exactly we verify at acceptance.
What well-structured device event logging delivers
Well-structured logging is not about "writing more to a file" but about a contract between the device, the application, and the server: which event counts as significant, in what order it is sent, what happens when the connection drops, and how receipt is confirmed.
Once this contract is fixed, the share of delivered events stops depending on channel quality: events are buffered on the device, tagged with a sequence number, and retried until confirmed.
In practice, this raises delivery from "hit or miss" to 99%+ even on an unstable channel, and losses during reconnection become isolated rather than systemic.
The second effect is predictable recovery time after a disconnection. If the log is written with checkpoints and, on restart, the application resumes from the last confirmed event rather than from the beginning, recovery takes seconds and requires no manual investigation.
The engineer can see from the log exactly which event the session broke on and understands the cause without reproducing it on a test bench.
The third is a controlled log volume. Without a retention policy, the log grows linearly and within a month eats up the device's memory, while the records you need can no longer be found in it.
Rotation by size and age, separate severity levels, and uploading only significant events to the server keep the volume within set bounds: diagnostics remain complete, and storage does not balloon.
The fourth is an eliminated class of failures. Most "mysterious" failures of IoT applications are not a loss of connectivity but unhandled edge cases: duplicate delivery of the same event, clock desynchronization, an event that arrives after a restart.
When these scenarios are covered by logging and checks, they move from "flaky" to diagnosable and are closed once, rather than coming back in every release.
Approaches to collecting and sending events: comparing modes
The logging mode is chosen not by preference, but by the device profile and the communication channel. Below are four working schemes that are actually chosen between at the start of a project: a continuous stream, batch sending, an offline buffer, and a hybrid. The table shows when each one fits and where it starts hitting limitations.
| Option | When it fits | Limitations |
|---|---|---|
| Continuous event stream | The device is always online, events are needed in real time (alarms, safety sensor telemetry) | Each event is a separate network exchange: on a weak channel, battery and traffic consumption grows, and when the connection drops, events are lost irretrievably |
| Batch sending | A regular stream of same-type events, a delay of minutes is acceptable | A buffer is needed on the device between sends; when the batch overflows, some events are discarded, and ordering requires explicit sorting by timestamp |
| Offline buffer | The connection is unstable or disappears for a long time: basements, warehouses, field equipment | Requires local storage with limited capacity and an eviction policy; during a long offline period, the oldest records may be lost |
| Hybrid scheme | Critical events are needed immediately, background telemetry — with a delay | Harder to maintain: two sending paths, priorities, separate retry delivery logic |
How the choice is made in practice
A continuous stream is justified only when the cost of delay is higher than the cost of traffic and battery.
For most devices, batch sending is more advantageous: events accumulate in a buffer and go out in a single exchange, which reduces the number of network sessions severalfold.
The offline buffer is not a separate mode, but insurance on top of the main one. A hybrid is assembled when both alarm events and background telemetry live in a single application: the former go out immediately, the latter — in batches on a schedule.
At the start, batch sending with an offline buffer is enough — the other modes are added when the load profile confirms it.
Stages of implementing logging in a mobile application
Implementing logging in a mobile application is not about connecting a library, but about designing the flow of events from the sensor to the storage.
Below is the sequence of work that we go through on every project: from auditing the current scheme to handover with documentation.
- Audit of the scheme. We look at how events are collected now: where they are lost when the network drops, whether they are duplicated on restart, and what happens when the app is minimized. The output is a map of bottlenecks and a list of scenarios that break.
- Designing the event format. We fix the mandatory fields: device identifier, timestamp, event type, payload, schema version. The version is mandatory — otherwise the very first extension of fields will break parsing on the server side.
- Setting up the sending queue. Events are written to local storage and sent in batches. The queue survives app restarts and network loss, and re-sending is rate-limited so as not to clog the channel when connectivity is restored.
- Protection against duplicates. Each event is assigned a unique key, and the server discards repeats on re-delivery. Without this, the same sensor signal is counted twice and distorts analytics.
- Testing scenarios. We run through real situations: network loss for several hours, an abrupt time zone change, background operation, bulk sending of accumulated data. We look at what reached the storage and in what order.
- Load testing. We assess how the queue behaves during prolonged accumulation and how many events per second the receiving side can handle.
- Handover. We provide the configuration, the event schema, the cleanup rules, and instructions for adding new event types.
After handover, the team can extend the set of events without our involvement — the format and rules are documented. Describe your task, and we will come back with an estimate and a work plan.
What the customer receives after implementation
We hand over not a "working application" but a complete package that your team can use to operate logging without us. Each artifact addresses its own class of tasks: deployment, incident investigation, policy changes, and knowledge transfer to new engineers.
- Event collection and dispatch configurations — files with logging levels by device type, buffering during connection loss, and retry rules. They can be changed without rebuilding the application.
- Event flow diagram — from the sensor through the mobile gateway to storage: where an event is buffered, where it is filtered, where it settles when the network is lost. It shows at which segment data is lost.
- Retention policies — lifetimes of raw and aggregated events, archiving rules, and the procedure for deletion after expiration. The registry stops growing uncontrollably while history for investigation remains.
- Documentation on formats and fields — a description of the event structure, required and optional fields, and schema versioning. A new developer connects a device without deciphering someone else's code.
- Incident investigation procedure — a step-by-step order: how to find a device's events by its identifier, how to distinguish a sensor failure from a communication loss, and to whom and within what timeframe to escalate.
- Dashboards and alerts — a snapshot of devices with anomalies: silent longer than normal, sending corrupted packets, lagging in time. They help spot a problem before users report it.
- Handover instructions — the procedure for accessing storage, rotating keys, and adding new device types. Your team introduces new equipment independently.
We hand over the package with a walkthrough on your side: engineers go through the flow diagram and the procedure on a real incident, not via a presentation.
Describe your task — we will come back with an estimate and a plan for what will be included in the handover specifically for you.
Case study: how we reduced event loss from devices
What it was before
The client came with a complaint that is hard to catch in tests: the devices were working, the app was showing status, but some events never reached the server.
The losses did not show up immediately — on long shifts, when switching between networks, and after waking from sleep mode. Engineers recorded a discrepancy between what the device logged and what the operator saw in the interface.
The investigation revealed three causes. The event buffer lived only in memory and was cleared when the process restarted. Sending was "single-threaded": if a request failed, the event was discarded instead of being retried.
And the log wrote everything indiscriminately — service pings, telemetry, user actions — into a single file without rotation, so when a failure occurred it was unclear exactly where the record was being lost.
What we changed
We moved the buffer to disk storage with a queue: an event is first recorded locally, then sent to the server, and only deleted after confirmation.
We added retries with increasing delay and deduplication by identifier — the network dropped, the event waited for connectivity and was sent once, without duplicates.
We split the log by levels and enabled rotation with a size limit so that old records would not displace fresh ones. We switched synchronization to batch sending with confirmation for each block — this way it is clear exactly which packet did not arrive.
What it became
The share of lost events dropped from 6-8% on long shifts to fractions of a percent, and on short sessions — to zero. The time for full synchronization after connectivity was restored was reduced by roughly a factor of three thanks to batch sending.
The log volume decreased severalfold: service noise was filtered out, and the necessary records are stored predictably.
A separate effect — incident investigation. Previously, the question "why did the event not arrive" ran into the absence of traces; now the log shows the path of each event: recorded, sent, confirmed.
We work on a professional mobile development stack, so we catch such failures at the design stage, not after launch.
How does an offline buffer differ from a continuous event stream?
An offline buffer and a continuous stream solve the same problem — getting an event from the device to the server — but they behave differently when connectivity drops.
In offline buffer mode, the application writes events to local storage and sends them in batches once the channel is restored. A continuous stream keeps the connection open and transmits each event immediately, relying on the stability of the connection in the moment.
The key difference is delivery latency. A continuous stream provides minimal latency: the event goes to the server the moment it occurs, which is critical for scenarios where real-time response matters.
An offline buffer deliberately sacrifices latency for resilience: when the connection drops, data is not lost but accumulates and is sent later. If the connection breaks frequently, a stream without buffering loses events, while a buffer does not.
The load on the device and the server is also distributed differently. A continuous stream keeps an active connection, consumes battery and traffic, and requires the server to maintain many simultaneous sessions.
An offline buffer saves resources: the device accumulates events locally and sends them less often, in larger packets, while the server receives them in portions.
The trade-off is peak load at the moment connectivity is restored, when the accumulated buffer is flushed all at once.
In practice, the choice depends on the class of events. For telemetry, where the trend matters rather than the exact second, an offline buffer is more reliable and cheaper in terms of resources.
For control commands and alarm signals, a continuous stream is needed — latency matters more than savings.
A hybrid is often used: critical events go through the stream, while background telemetry goes through a buffer with a configurable size and a policy for evicting old records.
A separate question is what to do with the buffer when it overflows and how to guarantee delivery. We build in idempotent event identifiers so that resending after a disconnection does not create duplicates, and a retention policy that limits the volume of local storage.
This makes it possible to maintain connection resilience without increasing memory consumption on the device.
Common technical concerns about IoT logging
What happens to the log if connectivity is lost for several hours?
Events are not lost: they are written to the device's local storage and sent to the server as soon as the channel is restored.
The send queue is size-limited — when it overflows, the oldest records are evicted, not the newest, because recent events matter more for incident investigation.
When connectivity returns, sending proceeds in batches with an increasing pause between attempts, so as not to overwhelm the channel or drain the battery.
How do you avoid duplicate events when resending?
Each event is assigned a unique identifier when created, and the server discards records with an already-known identifier. This is the key point: a delivery acknowledgement can be lost, the device will resend the event, and without such a check, copies would appear in the log.
Additionally, a sequence number is recorded within a session — it helps reconstruct the chronology even if events arrive out of order.
The log grows endlessly — how do you limit storage?
The device stores only a window of the most recent events, while the full history lives on the server. Locally, a rotation rule is applied: we keep a fixed volume or a time-based depth, and the rest is evicted.
On the server side, retention period and archiving are configured — old records move to cold storage, while the hot layer stays fast for searching. This way, volume does not grow linearly with the fleet of devices.
How do you make sure an event actually reached the server?
Delivery is confirmed at the protocol level: the device marks an event as sent only after the server responds, otherwise it returns it to the queue. For control, there is end-to-end verification — by identifier, you can trace the path of an event from the device to storage. If the share of confirmed sends drops, this is visible in monitoring before complaints appear.
What happens when the network changes or the application restarts?
The queue is persisted to disk, so restarting the application or switching from Wi-Fi to a mobile network does not discard unsent events. After the connection is restored, sending continues from where it stopped.
The only thing lost during a hard shutdown of the device is events that did not manage to be written to disk, which is why writing happens immediately when an event occurs, rather than being deferred.
Let's discuss your event logging task
To come back with an estimate and a plan, rather than generalities, we need baseline data on your logging environment. The more precise the inputs, the fewer assumptions in the estimate and the more realistic the timeline. Below is what to clarify and what materials to send.
- Event flow diagram: which devices and gateways write the log, over which transport (local storage, sync when network appears, always-on channel), and where records are currently lost when connectivity drops.
- Format and volume: record structure, event rate per device, daily and peak volume, retention depth on-device and on the server.
- Access model: who reads the log and why — support team, engineers, the user themselves; whether exports and role-based access control are needed.
- Mobile-side constraints: target platforms, minimum supported OS versions, requirements for background operation and battery consumption.
- Reliability requirements: acceptable loss rate, whether guaranteed delivery with acknowledgment is needed, behavior on buffer overflow.
- Privacy and storage: which fields are considered sensitive, whether anonymization is needed, retention periods and deletion-on-request rules.
- What already exists: current log collection code, schemas, access to the test bench, and samples of real records.
Send what you have — even a partial set will speed up the review. We'll come back with an estimate and a plan: we'll propose a collection and delivery architecture, define the stages, show where we build in headroom for losses, and how we verify the result on your bench. Describe the task — let's discuss the next step.