VR mode with stereoscopic rendering in mobile app

VR Mode Implementation in Mobile Apps (Stereoscopic Rendering)

Adding VR mode to existing mobile app isn't just splitting screen in half. Stereoscopic rendering requires correct projection setup per eye, distortion correction for specific optics, and head tracking without drift accumulation.

Stereo Rendering Geometry

Two eyes separated by IPD (interpupillary distance) — average 63–65mm. Each eye sees scene at slightly different angle — creates depth. For correct stereo effect render scene twice: cameras offset by ±IPD/2 along X from center point, but directed to common convergence point.

Projection matrix per eye is asymmetric frustum, not just offset symmetric frustum. Difference is fundamental: symmetric frustum creates "parallel eyes," asymmetric — realistic physiological convergence:

// Unity: asymmetric frustum for left eye
Matrix4x4 LeftEyeProjection(float ipd, float near, float far, float fov, float aspect) {
    float top = near * Mathf.Tan(fov * 0.5f * Mathf.Deg2Rad);
    float right = top * aspect;
    float shift = ipd * 0.5f * near / convergenceDistance;

    // left/right bounds shifted for specific eye
    return Matrix4x4.Frustum(-right + shift, right + shift, -top, top, near, far);
}

Lens Distortion Correction

VR headset lenses enlarge field of view but introduce barrel distortion — straight lines curve toward center. To compensate render with inverse pincushion distortion — result through distorted lens looks straight.

Distortion coefficients (k1, k2, k3) specific to each headset, encoded in Cardboard housing QR code. Google Cardboard SDK reads them automatically. For custom implementation:

// Fragment shader: barrel distortion
varying vec2 vTexCoord;
uniform float k1;
uniform float k2;
uniform sampler2D renderTexture;

void main() {
    vec2 coord = vTexCoord * 2.0 - 1.0; // [-1, 1]
    float r2 = dot(coord, coord);
    float distortion = 1.0 + k1 * r2 + k2 * r2 * r2;
    vec2 distorted = coord * distortion;
    vec2 uv = (distorted + 1.0) * 0.5;

    // Clamp + check bounds
    if (uv.x < 0.0 || uv.x > 1.0 || uv.y < 0.0 || uv.y > 1.0) {
        gl_FragColor = vec4(0.0, 0.0, 0.0, 1.0);
    } else {
        gl_FragColor = texture2D(renderTexture, uv);
    }
}

In practice use Cardboard SDK — it handles lens distortion, eliminating manual shader implementation.

Single Pass Stereo Rendering

Rendering each eye separately doubles draw calls. On mobile devices this is critical. Single Pass Instanced Rendering (SPIR) renders both eyes in one pass with instancing:

In Unity: Project Settings → XR Management → enable Single Pass Instanced. Works only with GL_EXT_multiview support (Android OpenGL ES 3.0+) or Metal with multi-view render targets (iOS 12+).

// Runtime support check
bool supportsSPIR = SystemInfo.supportsMultiviewRendering;
// If not — fallback to Multi-Pass

On 2019+ devices Single Pass Instanced works on most flagships. Budget Android devices with Mali GPU often don't support it.

Chromatic Aberration and Other Corrections

Through glass lenses image edges have chromatic aberration — red and blue channels shifted. Additional correction step shifts RGB channels based on distance from center. Cardboard SDK includes this in render pipeline automatically.

Mode Switching: Normal Screen / VR

App should work fine without headset. Switching:

// iOS
func toggleVRMode(enabled: Bool) {
    if enabled {
        startCardboardSession()
        UIApplication.shared.isIdleTimerDisabled = true // don't dim screen
        UIDevice.current.setValue(UIInterfaceOrientation.landscapeRight.rawValue,
                                  forKey: "orientation")
    } else {
        stopCardboardSession()
        UIApplication.shared.isIdleTimerDisabled = false
        // restore portrait
    }
}

In landscape for Cardboard: screen horizontal, Split-Screen vertical. iPhone notch needs handling — safeAreaInsets offset active zone.

Workflow

Audit existing app: content type, current render pipeline, target devices.

Integrate Cardboard SDK or custom stereo rendering pair.

Configure lens distortion, chromatic aberration correction.

Optimize: Single Pass Instanced, foveated rendering, LOD.

Adapt UI for VR mode: gaze interaction, reticle, Cardboard button.

Test latency and motion sickness on real devices.

Timeline Estimates

Add VR mode with Cardboard SDK to existing Unity app — 1–2 weeks. Custom implementation with native Metal/OpenGL, custom distortion shaders, full optimization — 3–6 weeks.