CRDT/OT Real-Time Synchronization

Implementing CRDT/OT for Real-time Synchronization on Website

Real-time multi-client sync without conflicts — harder than appears. Simple event broadcasting via WebSocket works until two users edit same fragment simultaneously. Without formal data model — chaos.

OT vs CRDT: Fundamental Difference

Operational Transformation (OT) — algorithm transforming operations accounting for concurrent changes. Used in Google Docs since 2006. Essence: if user A inserted symbol at position 5, user B deleted symbol at position 3, then A's operation must transform before applying to B.

Works correctly only with central server-arbiter ordering operations. Without it, OT for 2+ clients exponentially complex.

CRDT (Conflict-free Replicated Data Types) — data structures mathematically guaranteeing eventual consistency without coordination. Operations commutative and idempotent — order doesn't affect result.

CRDT in Practice: Yjs

Most mature CRDT library for browser and Node.js. Built from Y.Doc containing shared types: Y.Text, Y.Map, Y.Array.

npm install yjs y-websocket y-protocols

Server (y-websocket):

const { setupWSConnection } = require('y-websocket/bin/utils');
const http = require('http');
const { WebSocketServer } = require('ws');

const server = http.createServer();
const wss = new WebSocketServer({ server });

wss.on('connection', (ws, req) => {
  setupWSConnection(ws, req, {
    docName: req.url.slice(1),
    gc: true,
  });
});

server.listen(1234);

Client with editor integration:

import * as Y from 'yjs';
import { WebsocketProvider } from 'y-websocket';
import { QuillBinding } from 'y-quill';
import Quill from 'quill';

const ydoc = new Y.Doc();
const provider = new WebsocketProvider(
  'ws://localhost:1234',
  'my-document-room',
  ydoc,
  { connect: true }
);

provider.on('status', ({ status }) => {
  console.log('WS status:', status); // 'connected' | 'disconnected'
});

provider.on('sync', (isSynced: boolean) => {
  if (isSynced) console.log('Document synced from server');
});

const ytext = ydoc.getText('quill-content');
const quill = new Quill('#editor', { theme: 'snow' });
const binding = new QuillBinding(ytext, quill, provider.awareness);

provider.awareness.setLocalStateField('user', {
  name: 'Ivan Petrov',
  color: '#4a9eff',
});

Y.Map and Y.Array for Structured Data

Text editors — special case. CRDT applies wider: form state sync, kanban boards, diagrams.

// Synchronized task map (Kanban board)
const ytasks = ydoc.getMap<Y.Map<unknown>>('tasks');

function createTask(id: string, title: string, status: string) {
  const task = new Y.Map<unknown>();
  task.set('id', id);
  task.set('title', title);
  task.set('status', status);
  task.set('createdAt', Date.now());
  ytasks.set(id, task);
}

function moveTask(id: string, newStatus: string) {
  const task = ytasks.get(id);
  if (task) {
    task.set('status', newStatus);
  }
}

// Reactive UI update
ytasks.observe((event) => {
  event.changes.keys.forEach((change, key) => {
    if (change.action === 'add') {
      renderTask(ytasks.get(key));
    } else if (change.action === 'delete') {
      removeTaskFromUI(key);
    } else if (change.action === 'update') {
      updateTaskInUI(key, ytasks.get(key));
    }
  });
});

Persistence: Storing Y.Doc on Server

Ephemeral y-websocket loses document on restart. Production needs persistence layer.

// Option 1: y-leveldb (single-node)
const { LeveldbPersistence } = require('y-leveldb');
const persistence = new LeveldbPersistence('./data');

setupWSConnection(ws, req, {
  docName,
  gc: true,
  persistence,
});

// Option 2: Redis (multi-node via y-redis)
import { createRedisStorage } from 'y-redis';

const redisStorage = createRedisStorage({
  host: 'localhost',
  port: 6379,
});

// Option 3: PostgreSQL with Hocuspocus
import { Server } from '@hocuspocus/server';
import { Database } from '@hocuspocus/extension-database';
import { Pool } from 'pg';

const pool = new Pool({ connectionString: process.env.DATABASE_URL });

const server = Server.configure({
  port: 1234,
  extensions: [
    new Database({
      fetch: async ({ documentName }) => {
        const { rows } = await pool.query(
          'SELECT data FROM documents WHERE name = $1',
          [documentName]
        );
        return rows[0]?.data ?? null;
      },
      store: async ({ documentName, state }) => {
        await pool.query(
          `INSERT INTO documents (name, data, updated_at)
           VALUES ($1, $2, NOW())
           ON CONFLICT (name)
           DO UPDATE SET data = $2, updated_at = NOW()`,
          [documentName, Buffer.from(state)]
        );
      },
    }),
  ],
});

server.listen();

Automerge as Alternative

Automerge 2.x (Rust/WASM) better performance for large docs, native JSON structures:

import * as Automerge from '@automerge/automerge';

let doc = Automerge.init<{ items: string[] }>();

doc = Automerge.change(doc, 'Add item', (d) => {
  if (!d.items) d.items = [];
  d.items.push('new item');
});

const binary = Automerge.save(doc);

const [newDoc, patch] = Automerge.applyChanges(doc, [remoteChange]);

Conflict Resolution in CRDT

CRDT doesn't eliminate conflicts — determines deterministic winner. For Last-Write-Wins Map (LWW-Map), operation with later timestamp wins. For Yjs text, insertion position determined by neighbors, not index — stable with concurrent inserts.

Edge case: two users simultaneously delete and edit one element. Deletion wins in LWW, but Yjs keeps content as "tombstone" — allows correct application of changes made before deletion.

Scaling: Multi-Node Sync

One WebSocket server doesn't scale horizontally — each client connects to specific process. Solutions:

Sticky sessions on nginx level (document_id → upstream):

upstream yjs_backend {
    hash $arg_room consistent;
    server yjs1:1234;
    server yjs2:1234;
    server yjs3:1234;
}

Pub/Sub via Redis — each server publishes updates to Redis channel, others subscribed. Hocuspocus + Redis extension supports out of box.

Liveblocks/PartyKit — managed infrastructure for CRDT, if no desire maintain cluster.

Timeline

Basic CRDT text editor sync on Yjs + y-websocket: 3–5 days. Adding persistence via PostgreSQL/Redis: 2–3 more days. Multi-node with Redis pub/sub and load testing: plus 1 week. Custom data types (Kanban, diagrams) over Y.Map: depends on UI complexity.