AR portal development for iOS: virtual scene in a mobile app

AR portal on iOS: where exactly the virtual scene breaks

A failure begins where the virtual scene in the app first leaves the developer’s device. Different camera matrices, different sensor sets, and different physical screen dimensions mean that perspective, object scale, and anchor point are recalculated on each device.

If camera parameters and assumptions about scene size are hardcoded into assets for one device, on the others the model drifts: the object either hangs in the air or fits into frame only from a far distance.

In a demo this looks like “works only on one phone,” in production — like a failure on half the fleet.

The second typical failure is drift. The AR portal keeps an object relative to the real site through visual tracking, and tracking accumulates error every second.

Flat uniform surfaces, glare, glass display windows, and changing lighting provide few reference points: the scene starts correctly, but within half a minute of walking the anchor drifts by tens of centimeters, and the mockup stops matching the location.

This is treated not by a “more accurate algorithm” but by anchor architecture — starting markers, repeated plane estimation, anchor correction during movement.

The third failure is launch time. When the model, textures, and scene are pulled in at the moment of opening, the user holds the phone and waits tens of seconds, while the tracking session initializes until it captures a suitable frame.

Separating stages helps: a lightweight scene skeleton, asset preloading, loading details after start, and a separate waiting screen with a clear status — up to a hint of where to point the camera.

That is why we start AR portal development on iOS with a device matrix, calibration for each model, and a performance profile, rather than with a beautiful scene. Below we break down how each of these failures is addressed.

Measurable effect: scene launch, failures, repeat visits

After the scene is reworked, metrics are taken at three points: time until the first object appears on screen, share of failed launches, and share of repeat accesses to the scenario.

These show whether the AR portal retains the user or remains a one-off attraction that was opened once and closed.

From tap to first frame

The scene is divided into two levels: a lightweight skeleton with hints and loadable models. The skeleton appears in 1–2 seconds, heavy textures and animations load in the background while the user is already pointing the camera.

Scene load time decreased from 6–9 seconds to 1.5–2, and the share of abandoned AR scene launches — from 40% to 12%.

Failures and stability

The stability of a mobile app rests on three things: a geometry budget for a specific device, degradation of effects instead of failure, and restarting the session upon loss of recognition without leaving the app.

The share of app failures in scenarios with camera and 3D graphics fell from 8% of sessions to 0.5–1%. We collect failure statistics and pre-launch on a beta group via TestFlight, so that the numbers are confirmed not on a single demo device.

Returns to the scenario

Repeat openings of AR mode on a single device grew from 9% to 24% in the first month. Returns come from the object card and from a reminder — the scenario became part of the in-app journey.

We count specifically returns to the scene, not overall app launches: this shows that the portal holds interest on its own, without support from other sections.

AR portal operating modes: browser, app, offline

The scene delivery mode is chosen not by taste but by the site: where the AR portal will open, what device the visitor has, and whether there is connectivity on site.

The browser provides a link-based transition without installation, a native app — full access to the camera and a rich scene, an offline build — operation where the network is unstable. Each option has its own limitations, and they determine which mode becomes the primary one.

In practice, the modes do not compete: the scene is assembled once and delivered in several shells, and on weak devices effects are reduced all the way to flat viewing — audience reach is not lost.

That is why in the project we keep a common scene data layer and several delivery shells, and check compatibility on a device matrix — from old models to current ones.

The visitor gets the same scene regardless of the path by which they entered it, and the team does not rebuild content for each mode.

Implementation stages: from scene audit to handover of policies

Implementing an AR portal is structured differently from ordinary mobile app development: the physical site becomes part of the system, and an error in scene markup manifests on the visitor.

Therefore the first step is not assembly but an AR scene audit and recording the conditions in which the app will have to operate.

1. AR scene audit. We survey the site: lighting at different times of day, surface characteristics, elevation changes, corridor widths, camera behavior during movement. We record which objects must be recognized and from what distance — this is input data for the scheme.
2. Target scenarios and scene scheme. We describe the user route, anchor points of virtual objects, triggers, and fallback scenarios in case recognition does not work on the first try. The scheme is agreed before development begins — edits at this step cost an order of magnitude less.
3. App assembly. The portal is assembled as a separate module on top of the app interface: scene logic, content loading, camera operation, event analytics. We move content into a separate layer so that replacing the visual part does not require rebuilding the release.
4. Device testing. We test not only in the lab but also on a real fleet of models, including budget ones. We look at the share of successful scene launches, time to first frame, behavior in low light and while moving.
5. On-site verification. After installation we calibrate anchors, measure scene retention on a real flow of people, and adjust trigger thresholds. Without this step the app works stably only at the developer’s.
6. Handover and training. We hand over policies to the responsible persons: how to update content, what to do when recognition fails, how to collect diagnostics from the device. Plus brief training for site staff.
7. Post-launch support. We monitor the share of successful sessions by device and OS version, update content and the scene core as new versions of the mobile OS are released.

This order reduces rework: anchors and thresholds are fixed under live conditions, not in assumptions. We estimate timelines and scope for each stage after the audit — describe the site and the task, and we will come back with a plan.

Configurations, scene schemes, and content update policies

We hand over the project not as an archive of files but as a working system: your engineers can assemble a build, change the scene, and release an update without us.

Therefore the package includes not only source code but also configurations, scene schemes, and policies — with them a new person on the team gets up to speed in days, not weeks.

• AR portal configuration: parameter sets for internal builds, testing, and app store publication — which scenes and resources are included in each channel, which permissions are requested from the user.
• Scene schemes: the structure of each virtual scene — anchoring to markers and planes, object appearance points, resource loading order, and volume limits so the scene opens on an average device.
• AR scene documentation: how to add a model, replace a texture, adjust animation and anchor position, what file names and formats the build expects.
• Access policies: roles of editors and publishers, rights to modify the scene and roll back a version, action log for each change.
• Content update policy: the cycle “edit → device check → publish,” who approves changes, how an unsuccessful release is rolled back.
• Build and release instructions: sequence of steps, work with signing and key storage, checks before sending to the store.
• Project handover plan: walkthrough of schemes together with your team, joint release of the first update, and the point after which support follows your policy.

Case study: furniture showroom — scene launch reduced sevenfold

What it was before

The «Формат» furniture showroom brought a virtual showcase into a mobile app: a customer points a phone at a marker in the hall and enters a scene where the product stands in an interior.

The first AR portal build opened in 11 seconds, and on mid-range devices the launch failed in 28% of cases — the person saw a black screen and closed the app without ever reaching the product card.

What we changed and what the result was

We rebuilt the scene preparation pipeline: models are compressed and preloaded before entering the portal, textures are pulled in as the camera approaches, and rendering quality is selected by device class.

The portal is assembled from independent modules — marker, scene, purchase interface — so changing the collection does not affect the rest of the logic.

Metrics before and after:

• time to first frame — 11 seconds versus 1.6;
• share of failed launches — 28% versus 3%;
• share of users reaching an order — 4% versus 17%.

The virtual showcase stopped being a technology demonstration and became a working sales channel. The scene opens faster than the customer can lower the phone, failed launches dropped to the level of background failures, and the share of orders from the showcase grew fourfold.

What to do if the app is already built but the scene is unstable

A finished build and an unstable AR scene are often two different layers of the problem.

If the scene does not start on some devices or falls apart after a few seconds, the cause is usually not in the app code but in the content: overloaded models, heavy textures, no fallback scenario upon tracking loss. The first step is to break the build down by layers and understand where control is lost.

Many things can be fixed without rewriting the product: optimization of geometry and materials, splitting the scene into loadable parts, correct handling of tracking loss and return to the scene, calibration for weak devices.

Refining an existing app here is faster and cheaper than a full rebuild: the native part and interface remain, and only what hinders stability changes.

A rebuild is necessary where content is hardcoded into the build: the model sits in code, there is no separate scene loading layer, any edit requires publication in app stores.

Then we move AR content into a separate storage with versioning, add on-demand scene loading and a device cache — updating stops depending on releasing a new app version.

AR content migration proceeds in stages: first we transfer scenes without changing geometry, then we edit materials and anchor points, and finally we measure on what share of devices the scene initializes on the first try and how long launch takes.

For each block we decide separately — refinement or rebuild — and preserve the ability to roll back to a working version.

Technical concerns: compatibility, anchoring, updates

Before the start, clients ask the same set of questions: on whose devices this will work, how precisely the scene holds on the object, how to change content after release, and what remains on our side. Below are answers with defined boundaries, not a promise that “it works everywhere.”

How precisely does the scene hold on the object?

Anchor accuracy is determined by the reference: a plane, an image marker, or a pre-marked point.

For a static showcase a marker is enough; for walking around an object we place several references and refine the position — the scene does not drift away from the item and on repeat launch returns to the same place.

Can the scene be updated without releasing a new app?

Yes: content is separated into its own layer — models, texts, and step order are loaded at launch, without rebuilding the client and re-review in the store. If interaction logic changes, a new app version is needed; such edits are planned in advance.

How does the update reach users?

The current scene is cached on the device, so a repeat launch happens without delay and works offline. The new version is pulled in on the next connection, and the previous composition is retained — there is something to roll back to if the update produces an unexpected result.

What is included in portal support after launch?

We monitor the share of successful sessions, update content on request, and track platform changes so that the app does not fail after the next system update. Response times to a request and the responsible engineer are fixed before work starts.

What we clarify before the start: scenarios, devices, content

The first meeting on an AR portal is needed not for a presentation but to capture the inputs on which both the AR project estimate and the AR work plan depend. Until the site, device fleet, and list of objects are fixed, any timeline is an assumption, not a calculation. Below is what we clarify before the start.

• Site and conditions. Lighting, indoor or outdoor, area, uniform and reflective surfaces. This determines whether support markers are needed or plane recognition is sufficient.
• Device fleet. Which models are used by staff and clients, whether there is a depth sensor, what minimum OS version you are ready to support. This sets the boundary between a full virtual scene and a simplified mode.
• List of objects. How many models, in what form the source files will come, whether geometry and textures will need to be simplified for mobile rendering.
• AR use scenarios. Single display, object guide, try-on, staff training — the set of gestures and transition logic are built for a specific scenario.
• Who manages content. Whether content remains with the client, whether a control panel is needed, how scenes are updated without releasing a new app version.
• Offline and connectivity. Whether the portal works indoors without stable internet, whether models are cached on the device, what happens when the network is lost.
• Accesses, test points, and support. Test accounts, access to the real site for on-site verification, launch preparation mode, and the procedure for handing over updates.

The collected inputs are enough to come back with a stage estimate and scope of work.