Real-time collaborative drawing in mobile app

Real-time Collaborative Drawing in Mobile Applications

Collaborative drawing — even more demanding than whiteboard with shapes. Here each brush stroke is a stream of points with pressure, tilt and speed. Network latency perceived sharply: user expects other's drawing to appear simultaneously with their motion, not half a second later.

Division of Local and Remote Stroke

Key architectural decision: always draw local stroke immediately, bypassing network. Sync — for other clients, not sender.

// Flutter: local stroke
class DrawingBloc extends Bloc<DrawingEvent, DrawingState> {
  void onPointerDown(PointerDownEvent e) {
    currentStroke = Stroke(id: uuid(), points: [e.localPosition]);
    emit(state.copyWith(activeStroke: currentStroke));
    _syncService.beginStroke(currentStroke.id, color, brushSize);
  }

  void onPointerMove(PointerMoveEvent e) {
    currentStroke.points.add(e.localPosition);
    emit(state.copyWith(activeStroke: currentStroke));
    _syncService.appendPoints(currentStroke.id, [e.localPosition]);
  }

  void onPointerUp(PointerUpEvent e) {
    _syncService.finalizeStroke(currentStroke.id);
  }
}

Sync happens in parallel — local render doesn't wait for network.

Transport: Batching Points

60fps on mobile = 60 pointerMove events per second. With RTT 100ms batch for 100ms = ~10 points. Send batch every 50–80ms — balance between latency and traffic.

Message format (binary, not JSON — saves 3–5x by size):

[strokeId: 16 bytes UUID][pointCount: uint8][x1:f32][y1:f32][p1:f16][x2:f32]...

Float16 for pressure (pressure) — 0.0–1.0 with 0.001 precision sufficient. Float32 for coordinates (subpixel precision needed for scaling). Total ~10 bytes per point vs ~30 bytes in JSON.

On WebSocket: binary frames (ArrayBuffer in JS, Uint8List in Dart, ByteBuffer in Kotlin).

Smoothing Algorithm on Receiver Side

Remote points arrive batched with latency and discretely. Simple line drawing between points — stepwise. Need smoothing:

Catmull-Rom Spline — passes through all control points. For each pair of adjacent points generates intermediates. Suits drawing, needs no offline computations.

Perfect Freehand (Steve Ruiz library) — simulates brush shape accounting for pressure and speed, generates SVG-path from points. Works in Dart, JS, Swift. Result — organic curve, not just line with round cap.

For remote stroke: apply Perfect Freehand to entire accumulated point array on each update. Path redrawn fully. Costlier in CPU than incremental append, but visually correct (smoothing accounts for entire path context).

Layers and Object Order

Drawing without layers — primitive. Basic model: each stroke has z-index (creation timestamp). Concurrent drawing in one area — who drew last, stays on top.

Layers (layers) — optional feature. Each layer — separate Y.Array of objects. User picks active layer. Layer visibility/lock — field in Y.Map of layer.

On render: Canvas draws layers bottom-up. Each layer — separate offscreen canvas (iOS: UIGraphicsImageRenderer, Android: Bitmap with Canvas). Layers cached and redrawn only on change.

Eraser: Special Tool

Eraser doesn't draw white — it removes pixels. Two options:

1. Object-level eraser — deletes entire stroke on hit. Simple to implement, matches object model.
2. Pixel-level eraser — cuts stroke into parts. Requires geometric math (clip polygon by path).

For collaborative: object-level eraser simpler to sync (delete(strokeId) — atomic operation). Pixel-level — need to cut stroke and create new objects, harder in CRDT context.

Apple Pencil and Android Stylus

Apple Pencil via UITouch.type == .pencil:

• force — pressure 0.0–1.0
• altitudeAngle — tilt angle (0 = horizontal, π/2 = vertical)
• azimuthAngle(in:) — tilt direction
• predictedTouches(for:) — predicted future points

Flutter: PointerEvent.pressure, PointerEvent.tilt, PointerEvent.orientation — work for stylus on both platforms.

Syncing pressure/tilt to remote clients — worth it. Result visible: partner's brush shows same as they drew, with same dynamics.

Assessment

Collaborative drawing with basic tools (brush, eraser, color) on Flutter — 8–14 weeks. With stylus support, layers, pixel-level eraser and scalable canvas — 20–28 weeks.