AR Multiplayer Interaction in Mobile App

AR Multiplayer Interaction Development in Mobile Applications

Two people in one room see the same AR object from different angles. Sounds simple. In practice—synchronizing AR spaces of two devices, each in its own coordinate system, plus network latency, plus positioning drift. Without proper architecture, the first player's game piece is a meter away from where the second sees it.

Shared AR: Space Synchronization

ARKit—Collaborative Session. Since iOS 13, ARKit supports ARCollaborationData—binary packets with feature points and anchor info that devices exchange directly. Via MultipeerConnectivity (Bonjour, peer-to-peer Wi-Fi/Bluetooth) or custom relay server. Each device builds shared space map, objects on ARAnchor sync automatically.

// Send collaboration data to other participants
func session(_ session: ARSession, didOutputCollaborationData data: ARSession.CollaborationData) {
    let encoded = try? NSKeyedArchiver.archivedData(withRootObject: data,
                                                     requiringSecureCoding: true)
    sendToPeers(encoded)
}

Limitation: iOS–iOS only, no Android. For cross-platform, different solution needed.

ARCore—Cloud Anchors. Google provides server infrastructure: hostCloudAnchor() uploads anchor feature map to cloud, returns cloudAnchorId. Second user calls resolveCloudAnchor(cloudAnchorId)—gets same anchor in their space. Synchronize all shared object positions via it. Works iOS (ARCore SDK for iOS) ↔ Android.

Unity + Photon + ARDK (Niantic Lightship). For gamedev projects often better to use Niantic Lightship ARDK—native shared AR spaces support, multiplayer via Photon, cross-platform. But that's Unity, not native development.

Network Stack for AR Multiplayer

Real-time AR objects require low latency. Architecture:

Peer-to-peer (local, one network). MultipeerConnectivity (iOS) / Wi-Fi Aware (Android 8+) / Bluetooth GATT. 10–50 ms latency, no server. Great for tabletop games in one room, educational apps. Problem: NAT traversal transitioning to internet.

Relay server. WebSocket or WebRTC Data Channels via custom server or Firebase Realtime Database. 50–200 ms latency depending on geography. Firebase good for prototype, custom WebSocket needed at high load (>100 concurrent participants per session).

State synchronization. Two approaches: State Sync (server source of truth, all clients sync full state) and Delta Sync (transmit only changes). For AR objects with high position update frequency—Delta Sync with client-side interpolation (Entity Interpolation / Dead Reckoning).

Physics and Interaction Synchronization

If AR objects have physics (ball rolling, cubes falling)—need authoritative physics simulation. Options:

• Server-authoritative: server (headless) simulates physics, clients receive states and interpolate. Prevents cheating, requires powerful server
• Client prediction + rollback: client predicts result locally, server confirms or rolls back. More complex, better UX

For casual AR games, client-side physics with periodic state snapshot sync sufficient.

Testing AR Multiplayer

Cannot fully test in simulator—need real devices in real space. Set up CI/CD pipeline with Xcode Cloud / Firebase Test Lab, but final QA—physical, multiple devices in room. Metrics: object position latency between devices, anchor desynchronization, behavior on connection loss and recovery.

Timeline: basic shared AR with Cloud Anchors for two players in one location—4–7 weeks. Multiplayer AR game with server sync, multiple rooms, physics—3–6 months. Cost calculated individually.