VR gaze-based interaction in mobile app

VR Gaze Interaction in Mobile Apps

In mobile VR without controllers, gaze interface is the only way to interact with world. Seems simple: look at button, it activates. In practice, incorrectly implemented gaze frustrates user faster than any other UI pattern.

Gaze Raycast

Gaze is determined by camera view direction. Ray fires from camera position forward along Camera.main.transform.forward:

void FixedUpdate() {
    Ray gazeRay = new Ray(Camera.main.transform.position,
                          Camera.main.transform.forward);

    if (Physics.Raycast(gazeRay, out RaycastHit hit, maxGazeDistance, interactableLayer)) {
        var target = hit.collider.GetComponent<IGazeTarget>();
        if (target != null) {
            HandleGazeHit(target, hit.point);
        } else {
            HandleGazeMiss();
        }
    } else {
        HandleGazeMiss();
    }
}

FixedUpdate() instead of Update() — stable call frequency independent of FPS. On weak devices with 30 FPS dips, Update() gives uneven response.

interactableLayer — mandatory. Raycast across entire scene is expensive, and user shouldn't accidentally activate invisible colliders.

Reticle (Gaze Cursor)

Reticle is visual indicator of gaze point. Placed in world space on surface of object under gaze. Distance is dynamic: reticle "sticks" to hit point.

void UpdateReticle(Vector3 hitPoint, Vector3 hitNormal) {
    reticleTransform.position = hitPoint + hitNormal * RETICLE_OFFSET;
    reticleTransform.rotation = Quaternion.LookRotation(-hitNormal);

    // Constant scale in angular units (constant apparent size)
    float dist = Vector3.Distance(Camera.main.transform.position, hitPoint);
    reticleTransform.localScale = Vector3.one * dist * ANGULAR_SIZE;
}

When no object under gaze — reticle at default distance (3–5 meters). Don't hide it: user should always see where they're looking.

Dwell Activation and Progress Indicator

User looks at object for N seconds — activation happens. Optimal dwell time: 1.2–2.0 seconds. Less than 1 second — accidental activations during scene scan. More than 2 seconds — fatiguing.

Progress must be visible. Filling ring around reticle — standard:

public class GazeDwellController : MonoBehaviour {
    [SerializeField] private float dwellTime = 1.5f;
    [SerializeField] private Image progressRing;

    private float dwellProgress = 0f;
    private IGazeTarget currentTarget;
    private bool isActivated = false;

    public void OnGazeEnter(IGazeTarget target) {
        currentTarget = target;
        dwellProgress = 0f;
        isActivated = false;
        progressRing.gameObject.SetActive(true);
    }

    public void OnGazeStay() {
        if (isActivated) return;
        dwellProgress += Time.deltaTime / dwellTime;
        progressRing.fillAmount = dwellProgress;

        if (dwellProgress >= 1f) {
            isActivated = true;
            currentTarget?.OnGazeActivate();
            StartCoroutine(ResetAfterDelay(0.5f));
        }
    }

    public void OnGazeExit() {
        currentTarget = null;
        progressRing.gameObject.SetActive(false);
        dwellProgress = 0f;
    }
}

After activation — short cooldown before next activation of same object (0.5–1.0 sec). Otherwise user can't remove gaze in time and button "activates" twice.

Hover State: Feedback Before Activation

When user looks at object but dwell not complete, immediate visual feedback needed. Object should react on OnGazeEnter — before activation timeout. Options:

• Highlight: change material emission color
• Scale: object slightly enlarges (0.05f sufficient)
• Animation: icon reacts to gaze
• Sound signal: short click on dwell start

Without this user doesn't understand if app "sees" them.

Cardboard Button as Confirmation

Cardboard has physical button (magnetic trigger). Add it as alternative activation method instead of dwell — for advanced users faster and convenient:

// Cardboard SDK trigger event
void Update() {
    if (CardboardInput.GetButtonDown()) {
        TriggerCurrentGazeTarget();
    }
}

Button isn't dwell replacement, supplementary. Not all Cardboard housings have working magnetic button.

Common Implementation Mistakes

Too small collider on interactive object — user "misses" button. Collider should be 10–20% larger than visible object.

Activation triggers on any gaze pass over object — not only intentional fixation. Solved by minimum head angular velocity threshold at dwell start.

Reticle shakes from hand tremor — Cardboard SDK's gyro-fusion smooths this, but additional Lerp on reticle position (~20ms) removes residual shake.

Workflow

Analyze interactive elements: object types, interaction scenarios.

Implement raycast system with proper layers and colliders.

Reticle in world space with constant apparent size.

Dwell controller with progress indicator, hover state, cooldown.

Cardboard button as alternative trigger.

Test comfort: dwell time, button sizes, feedback.

Timeline Estimates

Basic gaze interaction system with reticle and dwell — 3–5 days. Full system with multiple interactive object types, animations, sound, configurable parameters — 1–2 weeks.