Developing an Online Video Editor: Browser-Based Editing and Rendering

Developing an Online Video Editor: Browser-Based Editing and Rendering

When a client requests browser-based video editing, the first hurdle is rendering the final output. Processing a one-minute clip on the client via FFmpeg.wasm works, but a ten-minute video will freeze even on a powerful laptop. We've tackled this on multiple projects and developed an architecture that scales. This article covers key architectural decisions: render type, timeline implementation, and file uploads.

Browser vs Server: Where to Render?

The choice between client-side and server-side rendering defines the entire stack and user experience. Here's a comparison:

Client-side rendering (WebCodecs API + FFmpeg.wasm): No server power needed; all load on the user's CPU. FFmpeg.wasm renders 10 minutes of video in 5–15 real-time minutes (depends on CPU). WebCodecs API is supported in Chrome 94+, Firefox 130+, and Safari 16.4+. Ideal for short clips up to 2 minutes—no delays, no server costs.

Server-side rendering (Remotion + FFmpeg): Rendering on GPU servers; the user gets a ready file via link. Remotion describes videos as React components, offering flexibility and reusability. Scales via AWS Lambda—rendering in parallel chunks. For a commercial product with videos longer than 2 minutes, this is the only reliable path. In our projects, we usually choose server-side: it gives predictable rendering times and doesn't drain the user's battery. But for a minimal editor for short clips, the client-side option is faster and cheaper.

Remotion official documentation: https://remotion.dev/docs/

How Is the Timeline Structured?

The central UI concept is a timeline with tracks. The data structure looks like this:

interface VideoProject { id: string; duration: number; // seconds fps: number; // 24 | 30 | 60 width: number; height: number; tracks: Track[]; } interface Track { id: string; type: 'video' | 'audio' | 'text' | 'image' | 'effect'; clips: Clip[]; muted: boolean; locked: boolean; volume: number; // 0–1 } interface Clip { id: string; trackId: string; assetId: string; // ссылка на загруженный файл startTime: number; // позиция на таймлайне (секунды) duration: number; // длительность клипа trimStart: number; // обрезка начала исходного файла trimEnd: number; // обрезка конца speed: number; // 0.25 – 4.0 opacity: number; transform?: ClipTransform; filters?: VideoFilter[]; } 

Data is stored in a state manager (Zustand or Redux) and synced with the database on save. Drag-and-drop is implemented via @dnd-kit—it reliably handles clip movement across tracks.

How to Split a Clip on the Timeline?

Split is a basic operation. Here's the step-by-step algorithm:

1. Determine the split point in seconds relative to the timeline (e.g., when dragging the marker).
2. Calculate splitPoint = (atTime - clip.startTime) + clip.trimStart.
3. Create two new clips: left with trimEnd = splitPoint and right with trimStart = splitPoint.
4. Delete the original clip and add the new ones.

Code:

const splitClip = (clipId: string, atTime: number) => { const clip = getClip(clipId); const splitPoint = atTime - clip.startTime + clip.trimStart; const leftClip: Clip = { ...clip, id: uuid(), duration: atTime - clip.startTime, trimEnd: splitPoint }; const rightClip: Clip = { ...clip, id: uuid(), startTime: atTime, duration: clip.startTime + clip.duration - atTime, trimStart: splitPoint, }; removeClip(clipId); addClip(leftClip); addClip(rightClip); }; 

Browser Preview

For preview, we use HTML5 <video> with synchronization via currentTime. Here's a key hook:

const PreviewPlayer: React.FC = () => { const { currentTime, isPlaying, tracks } = useEditorStore(); const videoRefs = useRef<Map<string, HTMLVideoElement>>(new Map()); useEffect(() => { // Синхронизируем все видео-клипы с таймлайном tracks.forEach(track => { track.clips.forEach(clip => { const video = videoRefs.current.get(clip.id); if (!video) return; const clipTime = currentTime - clip.startTime; const isActive = clipTime >= 0 && clipTime <= clip.duration; video.style.display = isActive ? 'block' : 'none'; if (isActive) { const targetTime = clip.trimStart + clipTime * clip.speed; if (Math.abs(video.currentTime - targetTime) > 0.05) { video.currentTime = targetTime; } isPlaying ? video.play() : video.pause(); } else { video.pause(); } }); }); }, [currentTime, isPlaying]); // ... }; 

Scrubbing on the timeline is done via mousedown/mousemove—calculating the position in seconds from the container width.

Clips: Drag, Trim, Split

Dragging: on drag, calculate the delta and update startTime considering pixels per second. Constrain to project boundaries and adjacent clips.

Trim: implemented similarly to split, but without deleting the original clip—just change trimStart/trimEnd.

Server-Side Rendering with Remotion

Remotion allows describing a video as a React component. We pass the project via inputProps and render on the server:

// Компонент для рендеринга const VideoComposition: React.FC<{ project: VideoProject }> = ({ project }) => { const frame = useCurrentFrame(); const { fps } = useVideoConfig(); const currentTime = frame / fps; return ( <AbsoluteFill style={{ background: '#000' }}> {project.tracks.flatMap(track => track.clips.map(clip => { const clipTime = currentTime - clip.startTime; if (clipTime < 0 || clipTime > clip.duration) return null; return ( <OffthreadVideo key={clip.id} src={clip.assetUrl} startFrom={Math.round(clip.trimStart * fps)} style={{ opacity: clip.opacity }} /> ); }) )} </AbsoluteFill> ); }; 

We launch rendering via the API using renderMediaOnLambda for scaling. On one project, we reduced the average rendering time for a 10-minute video from 15 minutes (client-side) to under 2 minutes using this approach.

Asset Upload

Large files (video, audio) are uploaded directly to S3 via presigned URLs—bypassing the application server:

// 1. Запрашиваем presigned URL const { uploadUrl, key } = await api.post('/api/editor/upload-url', { filename: file.name, contentType: file.type, size: file.size, }); // 2. Загружаем напрямую в S3 с прогрессом const xhr = new XMLHttpRequest(); xhr.upload.addEventListener('progress', (e) => { setProgress(Math.round(e.loaded / e.total * 100)); }); xhr.open('PUT', uploadUrl); xhr.setRequestHeader('Content-Type', file.type); xhr.send(file); 

This approach offloads the server and increases upload speed. Contact us to discuss your project details.

Our Process and Timeframes

We follow a structured process: data collection → audit/analysis → design → estimation → development → testing → launch. The minimum editor (timeline, trim, preview, export via Remotion) takes 13–17 business days. Full functionality with effects, transitions, and audio mixer takes 20–25 days. The final cost is determined after a brief.

What's included in the result:

• Architecture and project code with API documentation.
• Deployment instructions (Docker Compose, .env).
• Repository access and CI/CD.
• 12-month warranty on code and support during implementation.

Here's a breakdown of development stages:

We have 10+ years of experience and have completed over 40 projects in web development, including several online editors. Request a custom development—we will accommodate all your requirements.