How Subtitles Work in HLS: WebVTT Segments, X-TIMESTAMP-MAP, and Embedded Captions

Subtitles on an HLS stream almost never live inside the video file itself. They're their own manifest, their own segments, and — for anything delivered as WebVTT — their own small text format with rules that only make sense once you've seen how the player stitches them back onto a timeline that was never really "zero" to begin with. As covered in how HLS streaming works, video and audio are already separate fetches tied together by #EXT-X-MEDIA; subtitles use the exact same mechanism, which is why a stream can "have captions" in a way that has nothing to do with the segments a player is already downloading.

The subtitle rendition is just another #EXT-X-MEDIA entry

A master playlist advertises a subtitle track the same way it advertises an alternate audio track — a line like #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",LANGUAGE="en",URI="subs/en.m3u8",DEFAULT=YES. That URI resolves to a completely ordinary-looking media playlist — its own list of segment files, each a few seconds long, just like the video and audio playlists it sits beside. The difference is entirely in what the segments contain: instead of .ts or fragmented-MP4 media, each one is a WebVTT (.vtt) chunk covering that same time window. A player that ignores TYPE=SUBTITLES renditions loses nothing about video or audio playback — it just never learns the track exists, because nothing about the main variant stream references it.

What a WebVTT segment actually contains

WebVTT is a plain-text cue format: a WEBVTT header line, followed by blocks that pair a timestamp range with the text to show during it — something like 00:00:01.000 --> 00:00:04.500 on one line and the caption text on the next, repeated for every cue in the segment. The one line specific to HLS rather than WebVTT itself is X-TIMESTAMP-MAP, which appears right after the WEBVTT header — something like X-TIMESTAMP-MAP=MPEGTS:900000,LOCAL:00:00:00.000. Each subtitle segment's cues are timestamped from roughly zero, the same way video segments carry their own internal presentation timestamps, so X-TIMESTAMP-MAPis what tells the player how to translate "00:00:01.000" inside this particular segment into the stream's actual, continuous playback position (given here as an MPEG-2 TS clock value). Without it, a player stitching together dozens of subtitle segments would have no way to know where each one's zero point actually falls on the real timeline.

WebVTT sidecar tracks vs. burned-in CEA-608/708 captions

Not all captions travel this way. Broadcast-derived streams often carry CEA-608/708captions embedded directly inside the video elementary stream itself, as SEI (Supplemental Enhancement Information) messages attached to the H.264 or HEVC frames — there's no separate playlist entry at all, because the caption data rides along inside the same segment bytes the video decoder already reads. hls.js parses these out of the video stream in software (its bundled cea608-parser) and exposes them as in-band text tracks; a tool that only follows #EXT-X-MEDIA entries will never see them, since they were never listed as a separate rendition to begin with. WebVTT sidecar tracks and embedded CEA-608/708 captions solve the same visible problem through two structurally different delivery mechanisms, and a stream can carry either, both, or neither.

Why the browser needs explicit wiring, not just the segments

Downloading a subtitle segment doesn't make it appear on screen — the same distinction covered in why Chrome can't play HLS natively. Safari's native .m3u8 support reads TYPE=SUBTITLESrenditions itself and surfaces them in the system caption menu automatically. Everywhere else, it's hls.js's job: it fetches and parses each WebVTT segment, applies X-TIMESTAMP-MAP, and injects the cues into a real <track>element so the browser's own text-track rendering (positioning, line-wrapping, the built-in CC toggle) takes over from there. None of that happens implicitly just because the video is playing — a player has to specifically walk the master playlist's #EXT-X-MEDIA block, notice the SUBTITLES group, and choose to wire it up.

That's also the gap worth being upfront about: this site's own player and converter parse the master playlist for the highest-resolution variant and a default TYPE=AUDIO rendition, the same way the HLS playlist structure is described elsewhere on this site, but they don't currently follow a TYPE=SUBTITLESbranch. A source stream can advertise a WebVTT subtitle rendition and it simply won't be picked up — the resulting MP4 carries video and audio only, the same limitation any tool has if it isn't specifically parsing that part of the master playlist.

WebVTT vs. SRT

The other subtitle format worth knowing is SRT (SubRip) — an older, simpler cue format with a plain numeric index, a comma-based timestamp (00:00:01,000 --> 00:00:04,500), and the text, with no styling model at all. It predates HTML5 video and was never designed around a browser rendering it directly. WebVTT is effectively SRT's successor for the web: same core idea of timestamped text cues, but standardized by the W3C specifically for the <track> element, with cue settings for position and alignment and a small CSS-like styling hook (::cue) that SRT has no equivalent for. Browsers' native <track> support expects WebVTT; an .srt file has to be converted before a browser will render it as a text track at all, which is part of why HLS standardized on WebVTT for subtitle renditions rather than reusing SRT.

FAQ

If a stream has no #EXT-X-MEDIA TYPE=SUBTITLES entry, does that mean it has no captions? Not necessarily — it means it has no WebVTT sidecar track. The video could still carry CEA-608/708 captions burned into the elementary stream itself, which show up as an in-band text track a player extracts from the video, not as a separate playlist entry.

Can I just fetch the subtitle .m3u8 and download the .vtt segments directly? Yes — a subtitle rendition's media playlist and segments are fetched over plain HTTP(S) exactly like video and audio segments, subject to the same CORS rules covered in why CORS blocks video downloads. Concatenating the WebVTT segments in order and adjusting for each one's X-TIMESTAMP-MAP offset produces a single standalone .vtt file.

Does muxing a WebVTT track into an MP4 with ffmpeg just work? ffmpeg can mux WebVTT into an MP4 as a text or mov_textsubtitle stream, but it has to be given the assembled cues explicitly as an input — it won't discover and follow a TYPE=SUBTITLESrendition on its own just because it's pointed at the master playlist's video URL.