Developing AR Educational Content in Mobile Applications
Anatomy, molecular models, historical reconstructions, equipment schematics — educational AR works when object can be examined from all sides, disassembled, animated, and get interactive caption. Passive 3D model viewing without interaction — not education, it's pretty screensaver.
Task more complex than seems: combine AR tech with pedagogical logic, interaction scenarios and accessibility requirements.
Educational AR App Architecture
Typical structure: courses → lessons → AR activity. Each AR activity — separate scenario with own 3D asset set, animation script and interaction points (hotspots).
Hotspot is 3D annotation. Marker on carbon atom opens card "Carbon Atom: 6 protons, 6 neutrons". Implemented via BillboardComponent in RealityKit — marker always faces camera:
var billboard = BillboardComponent()
hotspotEntity.components[BillboardComponent.self] = billboard
Hotspot position — in 3D model's local coordinates. On animation (model disassembly) hotspot moves with part it's tied to. Make hotspot child entity relative to corresponding model part.
Interactive 3D Model Disassembly
Most requested mechanic in educational AR — "explosion" of model: parts scatter, exposing internal structure. Engine, heart, atom.
Implementation via AnimationPlaybackController and FromToByAnimation in RealityKit:
let explodeAnimation = FromToByAnimation<Transform>(
from: Transform(translation: [0, 0, 0]),
to: Transform(translation: [0.1, 0, 0]),
duration: 0.8,
timing: .easeInOut,
isAdditive: false,
bindTarget: .transform
)
let resource = try? AnimationResource.generate(with: explodeAnimation)
pistonEntity.playAnimation(resource, transitionDuration: 0.2)
For complex models with 50+ parts — pack animations in USDZ via USD Python API at content prep stage, not runtime generated.
Main Technical Problem: Content Size
Educational 3D models detailed. Anatomical body model — 200–500 MB in source. After mobile AR optimization — 20–50 MB. Loading on lesson start — user waits 30 seconds, closes app.
Solution: progressive loading + on-demand download. Basic app contains LOD-1 versions of all models (2–5 MB each). On entering AR activity — background download LOD-0 (detailed version). While loading — work with simplified. URLSession.downloadTask + local cache via FileManager:
let cachesURL = FileManager.default.urls(for: .cachesDirectory, in: .userDomainMask).first!
let modelURL = cachesURL.appendingPathComponent("\(modelId)_lod0.usdz")
// Check cache → if no → download → notify AR scene
Content server delivers models via CDN, split by lessons — not entire course at once.
Feedback and Knowledge Assessment in AR
Educational app without knowledge checking — just 3D atlas. Interactive tasks in AR:
- Drag-and-drop: drag organ to correct body location
- Sequence: assemble engine in correct part order
- Search: find and tap correct molecular structure
Drag-and-drop in AR non-trivial. EntityTranslationGestureRecognizer in RealityKit gives dragging on surface. But need snap-zone: when user releases organ near correct position — it "snaps". Implemented via CollisionComponent with trigger shape at destination zone and onCollisionBegan event.
Case
Molecular chemistry learning app, school segment grades 8–10. 50 molecules, each with disassembly to atoms and animation of chemical bond formation. Key requirement: classroom use — poor lighting, untextured desks (ARKit poorly detects planes). Solved via image marker (printed A4 sheet with marker on each desk) — ARImageTrackingConfiguration more stable than plane detection in challenging conditions. Markers distributed to teachers with QR code printout for download.
Timeline
| Content Volume | Timeline |
|---|---|
| 1–5 AR activities with ready 3D models | 3–5 weeks |
| Full course (10–20 lessons) with custom models | 3–5 months |
| Educational platform with content CMS | 5–8 months |
Cost calculated individually after analyzing pedagogical scenarios and content requirements.