How ffmpeg.wasm Converts Video in the Browser — and Why This Site's Converter Never Re-Encodes

Every other post on this blog has been about the M3U8 side of things — playlist tags, segment formats, the shape of the manifest a player or converter has to read. None of it explains the step that actually turns a pile of downloaded .ts or .m4s segments into the single .mp4 file /convert hands back. That step is FFmpeg — the same command-line tool used in countless server-side transcoding pipelines — except here it never touches a server. It runs as WebAssembly, inside the same browser tab that downloaded the segments, and it never leaves the machine it started on.

What ffmpeg.wasm actually is

FFmpeg is a C program. WebAssembly (WASM) is a portable, sandboxed bytecode format that browsers can run at close to native speed. ffmpeg.wasm is FFmpeg's C/C++ source compiled to WASM by Emscripten, packaged with a thin JavaScript API on top. The result is not a reimplementation or a subset of FFmpeg's behavior — it is the real FFmpeg codebase, built for a different target, running inside the same sandbox as the page's own JavaScript. That matters for a project like this one: every format quirk, every bitstream filter, every muxer FFmpeg supports on a server is available in the browser build too, gated only by which codecs the particular build was compiled with.

The trade-off is size. A native FFmpeg binary is already large; the WASM core this site loads (@ffmpeg/core@0.12.9) is roughly 30 MB, fetched once from a CDN and cached by the browser after that. That's why /convert's progress bar sits at the very start on a "loading ffmpeg" phase before a single video byte is fetched — the engine itself has to arrive before there's anything to run it on.

Why it runs in a worker, not the main thread

Decoding, demuxing, and remuxing video is CPU-heavy work, and doing it on the page's main thread would freeze the UI — the progress bar couldn't even repaint. ffmpeg.wasm sidesteps this by running the actual WASM module inside a Web Worker, communicating with the page over postMessage while the heavy computation happens off the main thread. That's a different worker, doing a different job, from the extension's content-script bridge described in how this site's privileged fetch bypasses CORS — one relays network requests through the extension, the other keeps a CPU-bound WASM module from blocking page rendering. Both happen to be workers; neither substitutes for the other.

The specific worker type matters too. This site's FFmpeg setup points classWorkerURL at a module worker (an ES module, not a classic script), because that's what the installed @ffmpeg/ffmpegpackage ships. Module workers can't call importScripts() — that API only exists for classic workers — so the FFmpeg core script loaded inside it has to be an ESM build, not the UMD build most CDN examples default to. Loading the UMD core into a module worker fails with an opaque "failed to import ffmpeg-core.js"error, which is exactly the kind of mismatch that's easy to hit by copying an older FFmpeg-in-the-browser tutorial without noticing the worker type changed underneath it.

Why this site's build is single-threaded

FFmpeg's WASM build comes in two flavors: a single-threaded core, and a multi-threaded core (@ffmpeg/core-mt) that parallelizes work using WASM threads backed by SharedArrayBuffer. Browsers only expose SharedArrayBuffer to a page that is cross-origin isolated — which requires the server to send Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policyresponse headers on every document. That's a hosting-level setting, not something a page can opt into with a meta tag.

This project is a Next.js static export — the whole website builds down to plain HTML, JS, and CSS files with no server process deciding what headers to send. There is no request handler anywhere in /convert's path that could attach COOP/COEP, so the multi-threaded core isn't an option here regardless of how the static files end up hosted. The single-threaded core has no such requirement — it runs under the ordinary same-origin policy any static file server already satisfies — which is the direct, load-bearing reason the converter uses it instead of the faster multi-threaded build.

Why /convert doesn't feel slow anyway

Losing multi-threading sounds like it should make conversion noticeably slower, but the FFmpeg invocation in components/Converter.tsx never asks it to decode or re-encode a single frame. The command it runs ends in -c copy— stream copy mode, which tells FFmpeg to repackage the existing encoded video and audio bitstreams into a new container without touching the compressed data at all. That's why /convertcan turn a multi-hundred-megabyte stream into an MP4 in seconds rather than minutes: it's doing the same class of work as unzipping and re-zipping a file, not the CPU-intensive work of decoding and re-encoding video, which is exactly where multi-threading would have mattered most.

Two smaller flags in that same command exist because a straight container swap isn't quite enough on its own. HLS audio is typically packaged as raw AAC with ADTS framing, but MP4's audio sample format expects AAC without those per-frame ADTS headers — so the command includes -bsf:a aac_adtstoasc, a bitstream filter that strips ADTS headers and writes the codec configuration MP4 expects, without decoding the audio itself. And -movflags +faststart moves the MP4's moov atom — the index of where every frame lives in the file — from the end of the file to the front, so a downloaded MP4 can start playing before the whole file has loaded, instead of requiring a seek to the end first.

Where the actual video data comes from

None of this explains how the segments got onto the WASM module's virtual filesystem in the first place. Before ffmpeg.exec() ever runs, stageMedia() downloads every segment named in the parsed media playlist — plus the AES key and initialization segment if the playlist has them — through the same bridge-first fetchBytes()used everywhere else on this site, and writes each one into FFmpeg's in-memory filesystem with ffmpeg.writeFile(). It also rewrites a local copy of the playlist so every URI points at one of those in-memory filenames instead of the original remote URL. FFmpeg never makes a network request of its own — by the time it runs, the entire multi-file playlist and every segment it references already exist as local files inside the WASM sandbox, and -allowed_extensions ALL -protocol_whitelist file,crypto,datais what tells FFmpeg's own HLS demuxer it's allowed to read a playlist referencing local files at all — FFmpeg's built-in HLS support otherwise assumes it's being pointed at a live URL, not a virtual filesystem.

What this design can't do

Stream copy has one hard requirement: the source and destination container both have to support the codec as-is, because no decoding happens to convert between codecs. A stream encoded in a codec MP4 can't carry unmodified would fail this remux, and there's no fallback path in /convert that switches to re-encoding — it either copies cleanly or the ffmpeg.exec() call fails outright. That's also, more subtly, why /convertcan't trim a clip to an arbitrary time range the way a planned /trim feature would need to: cutting on an exact timestamp usually requires seeking to the nearest keyframe or decoding into the middle of a GOP, and a pure -c copy mux has no decoded frames to cut at that precision.

FAQ

Does converting a video ever re-encode it? No — /convert's FFmpeg command always runs with -c copy, so the original video and audio bitstreams pass through unchanged. Quality is identical to the source; only the container changes.

Why does the converter need to download ~30 MB before it does anything?That's the FFmpeg WASM core itself, not video data — it's fetched once per browser session (the browser caches it afterward) before any segment download begins.

Could this site ever ship the faster multi-threaded FFmpeg core? Only if it stopped being a static export and moved to a host that can attach COOP/COEP response headers to every document — SharedArrayBuffer, which the multi-threaded core depends on, isn't available to a page without them, regardless of how fast the visitor's CPU is.