ADR 0012: Video transformations run in their own subsystem over FFmpeg filter graphs

• Status: accepted (WP-120; owner decision 2026-10-04); reconciled with the WP-121 implementation on 2026-10-04 (see “Reconciliation with WP-121”); URL syntax and audio added by WP-122 and WP-123 (last section); HLS and DASH (§6) as implemented by WP-126 (2026-10-05); cost-based admission by WP-121c (2026-10-06, its section); spool files (§5 and their section) by WP-121b (2026-10-06); video presets and the restricted unsigned video policy by WP-132 (2026-10-15, §8 and “Video presets”); MPEG-TS segments, overlays on streams and chunked stream audio by WP-131 (2026-10-15, §6 and “HLS extras”)
• Date: 2026-10-04

Context

Until M4 video is an input of the image pipeline: a frame, a sprite or an animation is decoded by internal/video/ffmpeg (ADR 0010) and processed by libvips, and an animated result can be encoded to MP4/WebM from RGB frames (WP-76). The owner (2026-10-04) asked for video transformations comparable to Cloudinary video and Cloudflare Media Transformations / Stream inside this project:

• Phase A, short synchronous transforms: trim, resize/crop/fit, fps, rotate/flip, quality/bitrate, MP4/WebM output, mute, audio-only output (M4A/AAC, Opus, MP3), image and text watermarks; poster frames and sprites already exist (WP-74/75). Results are materialised, cached and served with HTTP Range.
• Phase B, HLS and DASH for VOD with just-in-time segment transcoding and adaptive-bitrate (ABR) ladders.
• Same binary and the same full/gpl images (slim stays without video), a separate subsystem with its own worker pool and limits, off by default, deployable as a dedicated role (XI_ROLE=all|image|video).
• Out of scope: live streaming, upload/catalogue management (a media library), DRM.

Video work differs from image work in every dimension that the image pipeline’s design depends on: a job takes seconds to minutes instead of milliseconds, its input and output are tens to hundreds of megabytes, it has an audio track, the output is consumed with Range requests by players, and most of the cost is in the encoder. Re-using the image path (libvips per frame, the image semaphore, in-memory results) would let one video request starve the image service and hold whole videos in memory.

Decision

1. Packages

• internal/video/ffmpeg stays the only cgo package besides internal/vips (ADR 0010, CLAUDE.md rule 3 as amended there), behind the ffmpeg build tag. It gains a transcoder: transcode.c (an opaque job handle that demuxes, decodes, runs one libavfilter chain per stream, encodes and muxes, video and audio, in one Run call) and output.c (the growable output buffer shared with the WP-76 animation encoder, with the +faststart/cues_to_front rewrite through io_open). No FFmpeg type leaves the package; the stub answers Unsupported without the tag.
• The request model is transform.VideoOp on a transform.Request of kind transform.KindVideo (cut, geometry, rotation/flip, frame rate, container and codecs, quality or bit-rate cap, audio on/off and bit rate), so dialects map their syntax onto the same transform.Request they already build for images, and the canonical key, signing and the unsigned policy apply unchanged. Every field of the transcode is a number, an enum or a colour; the only strings are an overlay’s source and Pango text (WP-124), which libvips renders into pixels and which never reach FFmpeg (§3).
• The planner is cgo-free in internal/video: TranscodeCaps.Resolve (container → muxer and codecs on this build, before any fetch) and PlanTranscode (cut, geometry, rates, encoder settings and the filter chains, from the probe).
• The subsystem is internal/media (pure Go; depguard forbids vips, video/ffmpeg and server): Service owns the admission pool, coalescing, the timeout and the metrics, over an Engine that internal/app implements with video/ffmpeg. The HTTP side (role check, unsigned policy, cache, ETag, Range) is internal/server/transcode.go, which sees the subsystem as server.VideoTransformer. (The working names internal/video/transcode and video.Transform of the first draft were not used.)
• The image pipeline keeps posters, sprites, animations and GIF → video (WP-74/75/76); the video subsystem does not route through libvips except to render overlay images (§3).

2. Pipeline: FFmpeg end to end, not libvips

decode (libavformat + libavcodec) → one libavfilter graph (video and audio chains) → encode (libavcodec) → mux (libavformat), streamed frame by frame. libvips is not used per frame: a YUV → RGB → libvips → RGB → YUV round trip costs more than the transform for every option on the Phase A list (scale, crop, pad, fps, transpose, overlay are native, SIMD, YUV filters), libvips has no audio, and a per-frame libvips call would put the image library inside the encoder’s loop.

• No request text reaches the graph. The planner (video.PlanTranscode) writes each stream’s chain from a fixed vocabulary of filter and option names (scale, crop, pad, fps, trim/atrim + setpts/asetpts, transpose/hflip/vflip, format, aresample, …) and numbers formatted with strconv; the shim creates the buffer/abuffer source and the sink itself (avfilter_graph_alloc_filter, options through AVOption, configured from the decoder) and links that chain between them with avfilter_graph_parse_ptr. The guarantee rests on two things: the chains are written only from transform.VideoOp’s numeric fields (enums, integers, floats and a colour, all validated; an overlay’s source and text become pixels, §3), so nothing a request says can be copied into the chain, and TestFilterChainsAreNumeric fails if a chain ever holds a character outside the filter-name/number alphabet (a–z _ 0–9 = : , . - + / and PTS/B). This replaces the first draft’s one-node-at-a-time AVOption graph: for numeric options it is equally safe, it is a few lines of C instead of a node builder per filter, the plan is testable in pure Go, and the chain is readable in logs and tests. The rule for later options: text never goes into a chain. Anything carrying text (WP-124’s drawtext timecode expansion and its expressions) is set by the shim as an AVOption value on a filter it creates with avfilter_graph_alloc_filter, and the alphabet test stays (WP-124 added TestOverlayGraphIsNumeric for the overlay expressions).
• Trim seeks to the keyframe before the start and drops frames up to it (exact trims; -ss style fast keyframe trims only when the request asks for them), so a 10 s clip of a 1 h source decodes about one GOP more than the clip.
• Geometry follows the image pipeline’s semantics (fit, fill, crop with gravity, pad with a colour, display matrix applied before anything else) and its option names where they exist; even dimensions for 4:2:0 output are enforced by the planner, not by the encoder failing.
• Encoding uses the encoders the base image has (§7), chosen at startup by probing (avcodec_find_encoder_by_name, as WP-76), with CRF/quality or target bitrate (+ maxrate/bufsize), a keyframe interval in seconds, and XI_VIDEO_TRANSFORM_THREADS codec threads per job instead of WP-76’s single thread: a transcode is one long job, not many short ones.
• Audio: decoded, resampled by libswresample through the aresample filter (the shim calls no swr_* function, so the -tags ffmpeg pkg-config line names libavfilter, not libswresample), mixed to mono/stereo, optionally normalised (loudnorm, single pass) and encoded; muted outputs drop the stream; audio-only outputs drop video.

3. Overlays

As implemented by WP-124 (user-facing semantics in FORMATS §5.2 “Overlays”):

• Model. transform.VideoOverlay embeds the image transform.Watermark (source, Pango text, font, colour, opacity, gravity and offsets, scale or size box, rotation, shadow, tiling) and adds a window (Start, End) and fades (FadeIn, FadeOut) on the output timeline, or a Timecode format; at most 8 per request, never on audio-only outputs. Dialects map their watermark options onto the same struct for images and videos. The overlay’s source and text are the only strings a VideoOp carries; neither ever reaches FFmpeg.
• Image watermarks are fetched through the normal source path (server.fetchImages, shared with wmu: allowlists, the SSRF-safe fetcher, size limits, SVG sanitising), after the cache lookup and only when a transcode runs, held for the coalesced job like the source. Inside the job (the video worker slot and timeout), once the plan knows the output size, exec.Executor.RenderOverlay runs the image pipeline’s watermark step on a frame of that size (decode, fit, rotate, shadow, opacity × XI_PROCESSING_WATERMARK_OPACITY, gravity or tiling, clip to the frame) and writes straight 8-bit RGBA into C memory. One decoder (libvips) for every watermark image, no FFmpeg image decoder on untrusted overlays.
• Static text is rendered the same way, with the libvips/Pango text watermark code and the security/pango sanitiser, so text looks identical on images and videos (tested pixel for pixel against the image path) and uses the same fonts, shaping and size limits. Hostile text is just glyphs: %{…} or graph syntax is drawn literally.
• Graph. The shim builds a second graph after the stream’s chain (whose sink then feeds it), filter by filter with avfilter_graph_alloc_filter and AVOption values, nothing parsed: buffer (the chain’s frames) → per image overlay overlay (x, y, eof_action=repeat, enable) whose second input is a buffer source of the RGBA pixels, through fade filters (alpha=1, start and duration as AVOption durations) when it fades → per timecode drawtext → format (the chain’s 4:2:0) → buffersink. Every frame enters the second graph with its final timestamp, so the shim pushes an overlay’s frame with the first video frame’s timestamp (the overlay filter repeats it) or, while it fades, with every frame’s. The pixels are copied into a frame during xi_tx_run; C keeps no pointer to memory Go owns.
• Timeline expressions (between(t,a,b), gte(t,a), lte(t,b)), drawtext positions (w-tw-10, (h-th)/2+0) and alphas (0.8*clip(min((t-1)/0.5,(3-t)/0.25),0,1)) are written by the planner (video.planOverlays) from numbers and a fixed vocabulary; TestOverlayGraphIsNumeric checks their alphabet and names whatever the overlay’s text and source say, and the chain alphabet test still holds.
• Dynamic text is the timecode only: FFmpeg’s drawtext (FreeType + HarfBuzz + FriBidi + Fontconfig) with expansion=normal on a constant the planner picks from the enum (%{pts:hms}, %{pts:flt}), the default Fontconfig family (Sans), fontfile/textfile never set, size, colour, opacity, shadow and a compass position from numbers. No request text is ever a drawtext text.
• Capabilities. TranscodeCaps.Overlays (overlay, fade, format) and .Timecode (drawtext) are probed at startup; Resolve answers 422 for overlays the image cannot draw before anything is fetched. Memory: one RGBA buffer of at most the output frame per image or text overlay for the job’s duration (with the frame copy in C, twice that while it runs).

4. Concurrency, limits and roles

• Own pool. Admission of its own (internal/media, x/sync semaphores; WP-121 used a second internal/limiter instance, replaced by cost-based admission in WP-121c, see “Cost-based admission” below): XI_VIDEO_TRANSFORM_WORKERS (default max(1, GOMAXPROCS/4)) sizes it, transcodes are charged their estimated cost against XI_VIDEO_TRANSFORM_CAPACITY with at most XI_VIDEO_TRANSFORM_MAX_JOBS running, XI_VIDEO_TRANSFORM_QUEUE waiting jobs and _QUEUE_COST waiting cost (fail fast: 429 + Retry-After), XI_VIDEO_TRANSFORM_QUEUE_TIMEOUT (503), and XI_VIDEO_TRANSFORM_THREADS codec threads per job (default 1). Every video-transformation setting is under XI_VIDEO_TRANSFORM_*, apart from the XI_VIDEO_MAX_*/XI_VIDEO_ENABLED keys of video sources for thumbnails (WP-74), so the two sets of limits can differ (a transcode’s source is usually larger than a poster’s). The image semaphore is never taken by a video job and vice versa, so a burst of video work cannot raise image latency beyond the CPU it shares (and with the role split, not even that).
• Coalescing. Identical jobs (same cache key, §5) run once; later requests wait on the running job. A job’s context is detached from any single request and cancelled when its last waiter leaves or XI_VIDEO_TRANSFORM_TIMEOUT expires (504) (rule 8: ctx.Err() is checked between packets through the existing abort flag).
• Time. A video request’s budget is XI_FETCH_TIMEOUT + XI_VIDEO_TRANSFORM_QUEUE_TIMEOUT + XI_VIDEO_TRANSFORM_TIMEOUT instead of XI_SERVER_REQUEST_TIMEOUT, and its write deadline is extended to match with http.ResponseController (the server-wide WriteTimeout stays sized for images); a client waits for the job, then gets the materialised file.
• Roles. XI_ROLE=all (default) serves both; image never creates the video pool and video transformations answer 404 (so a load balancer sends them elsewhere); video serves only video transformations plus /healthz, /readyz and the admin listener (metrics, pprof), and answers every image, /info and raw request 404; it requires XI_VIDEO_TRANSFORM_ENABLED=true (config validation). Where the role allows them but transformations are disabled, they answer 404 too. The routes are the same on every role: the role decides after parsing, by the request’s kind. One binary and one config schema: an operator routes /video/*-style paths (or a dialect’s video endpoints, and /_stream/*, the parts of HLS and DASH streams, §6) to a video deployment and keeps a C crash in a demuxer or encoder away from the image fleet.
• Off by default. XI_VIDEO_TRANSFORM_ENABLED=false (a plain boolean, no auto: the feature must be asked for); true fails startup on an image without FFmpeg (slim), whatever the role. An output whose encoder the image lacks (MP4/H.264 on full, HEVC anywhere but gpl) is a per-request 422 from TranscodeCaps.Resolve before anything is fetched, and the startup log lists the outputs the build can write. It is independent of XI_VIDEO_ENABLED (video sources for thumbnails).

5. Materialise, cache, serve with Range

• Phase A does not stream an encode to the client: MP4 needs its moov box at the front (+faststart) for progressive playback and players issue Range requests that need a known length. The job encodes into C memory (the growable buffer of output.c, through the shim’s custom write/seek AVIO; no FFmpeg protocol; the faststart and WebM cues_to_front passes rewrite it through io_open as WP-76 does), stopped by the muxer at XI_VIDEO_TRANSFORM_MAX_OUTPUT_BYTES, handed to Go without a copy as a refcounted buffer, committed to the result cache and served with http.ServeContent: Range (multipart too), If-Range, 206, 416, ETag, Accept-Ranges: bytes. Since WP-121b an output that grows past XI_VIDEO_TRANSFORM_SPOOL_THRESHOLD moves into a spool file instead (last section).
• Sources up to XI_VIDEO_TRANSFORM_SPOOL_THRESHOLD (32 MiB) are fetched into C memory (as image sources are) and read by the shim through a read/seek AVIO over it; larger ones (up to XI_VIDEO_TRANSFORM_MAX_INPUT_BYTES) are downloaded into a spool file and read through the same AVIO with pread on a duplicated descriptor (WP-121b, last section). The WP-121 bound of XI_VIDEO_TRANSFORM_WORKERS × (MAX_INPUT_BYTES + MAX_OUTPUT_BYTES) of memory (defaults: workers × 768 MiB) is now about MAX_JOBS × 2 × threshold (plus QUEUE opened sources of up to the threshold, WP-121c) and the codecs’ working memory; the disk holds the rest, bounded by XI_VIDEO_TRANSFORM_SPOOL_MAX_BYTES. With the threshold at 0 the WP-121 behaviour (everything in memory) is kept.
• Cache. The key is the canonical request (its VideoOp) + source identity + the resolved output (v:<container>:<video codec>:<audio codec>, so the same URL producing AV1 in full and H.264 in gpl never shares an entry) + the build. Video results use the image result cache’s tiers. Since WP-121b a spooled result is streamed from its file into the disk and S3 tiers (the memory tier skips it, redis keeps values up to its limit), and a disk hit of at least 8 MiB is a file-backed body over the entry file, so a Range request reads only its range; an S3 hit is still read whole (a ranged GET is future work), and Tiered copies it into the disk tier in front of it.

6. Phase B: HLS and DASH by just-in-time segments

As implemented by WP-126 (2026-10-05; user-facing semantics in URL-DIALECTS §3.10 and FORMATS §5.2 “HLS and DASH streams”).

• Model. transform.VideoOp.Stream (transform.VideoStream: protocol HLS or DASH, rendition heights, min/max height and a top-N bound for Cloudinary’s profiles, segment duration in whole seconds). Everything else of the VideoOp applies to every rendition (cut, crop, rotation, flips, frame rate, quality, codec, keyframe interval, fades, audio options); validation refuses what a stream cannot hold (sizes and bit rates are the renditions’, the container is CMAF, or MPEG-TS for HLS with H.264, HEVC is not streamed, the audio is AAC at 48 kHz). Overlays are drawn on every rendition since WP-131 (“HLS extras”). The stream is part of the canonical key, so an HLS and a DASH manifest never share a cache entry.
• Manifests from the probe alone. video.PlanStream lays a stream out from the probe (TranscodeInput) and the configuration: the rungs (the URL’s, else XI_VIDEO_TRANSFORM_ABR_LADDER, by the short side of the frame; rungs larger than the source dropped, one rendition of the source’s size when none fits), each rendition’s size (the source’s aspect after orientation and crop, rounded to even), one constant frame rate for all (the source’s or the request’s, capped by XI_VIDEO_TRANSFORM_MAX_FPS, as a fraction: NTSC rates are n·1000/1001), the segment grid (XI_VIDEO_TRANSFORM_SEGMENT_DURATION, default 6 s, whole seconds; the last segment exists when it holds half a frame) and the codec strings. A source of unknown duration cannot be streamed (422). media/manifest writes HLS version 7 (master with EXT-X-STREAM-INF per rendition and the audio as an EXT-X-MEDIA group; media playlists with EXT-X-MAP, EXT-X-PLAYLIST-TYPE:VOD, EXT-X-INDEPENDENT-SEGMENTS, exact EXTINFs on the frame grid) and a static DASH MPD (isoff-live profile, one SegmentTemplate per adaptation set with $Number$ from 0 and presentationTimeOffset). The probe is cached; a manifest costs no transcode.
• Codec strings before encoding. H.264 is told its level (c->level, the lowest of ITU-T H.264 Table A-1 from 3.0 up that holds the frame size, macroblock rate and the rung’s bit rate) and the manifest names High profile at it; AV1’s seq_level_idx and VP9’s level are what libaom and FFmpeg’s vpcc.c derive (AV1 Annex A picture size and display rate; VP9 picture size only, since the MP4 muxer passes vpcc.c no frame rate), computed the same way. TestStreamCodecStrings encodes renditions up to 1080p60 in each codec and compares the init segment’s avcC/av1C/vpcC with the manifest’s string. AAC-LC is mp4a.40.2.
• Video segments on demand. A segment request transcodes [n·d, (n+1)·d) of the stream as its own job (video.PlanSegment): the demuxer seeks to the keyframe before n·d and stops one second of packets past (n+1)·d (a segment decodes its window plus at most one source GOP), trim keeps the source frames of the window, setpts keeps the stream’s clock (0 is the stream’s start for every segment), then orientation, crop and the scale to the rendition, then fps=fps=N/D:start_time=G with G the first point of the stream’s frame grid in the window and trim=end at the window’s end, so every frame of the stream belongs to exactly one segment whatever the source’s own timestamps. A new encoder per segment starts with an IDR/keyframe and closed GOPs (one GOP per segment, or ki long). The output is CMAF fragmented MP4 written by the shim with empty_moov+delay_moov+default_base_moof+frag_custom+skip_trailer+frag_discont, use_editlist=0 and avoid_negative_ts=disabled into the same growable buffer (one fragment per segment; delay_moov because libaom’s av1C only arrives with its first packet); Go cuts it into the init segment and the media segment (media/fmp4), and the init segments of all segments of a rendition are byte-identical (tested), so the one from segment 0 serves. No multi-file io_open is needed.
• One timeline, non-negative composition offsets. frag_discont makes each fragment’s tfdt its first sample’s decode time instead of 0, so independently encoded segments join without a gap or an overlap (tested: the joined segments of every rendition decode to the source’s frame count on the grid). x264’s B-frames make the first decode time precede the first presentation time; negative composition offsets (CMAF’s trun v1) would avoid a shift, but FFmpeg’s demuxer then presents the video late by the reordering delay while browsers do not, so instead every timestamp of a stream (each rendition and the audio) is moved by one stream offset of at least the AAC priming (1024 samples) plus the H.264 reordering delay (2 frames), rounded to 15 ticks so it is exact at 90 kHz and 48 kHz (about 0.1 s; 0.021 s without H.264). The video track’s time base is 90 kHz. HLS players align on the first fragment; the MPD’s presentationTimeOffset removes the offset.
• Audio is AAC-LC 48 kHz with the request’s audio options, fragmented on the segment grid: the shim flushes a fragment before the first packet at or after every k·d of the source, so audio segment k is fragment k of the track; the encoder’s priming exists once, at the start, and no boundary has a gap, an overlap or a click (tested: packet timestamps continuous across the joined segments, the track’s length the stream’s, the first video frame plays with the source’s first sample). WP-126 encoded the whole track at once (video.PlanAudioTrack); since WP-131 it is encoded in chunks of whole segments that join as that single encode would (video.PlanAudioChunk, “HLS extras”), and every audio segment is its own cache entry.
• Child URLs and their signing. The manifest is answered at the dialect’s URL (signed like any video transformation: signed only by default, roles, the unsigned policy). Everything it names lives under <path prefix>/_stream/<ticket>/ (master.m3u8, manifest.mpd, v<i>.m3u8, a.m3u8, v<i>/init.mp4, v<i>/<n>.m4s, a/…), outside every dialect, so a player fetches it with no signing key. The ticket (security/ticket) is base64url(1 ‖ nonce ‖ AES-256-GCM(k_enc, nonce, expiry ‖ payload)), the payload the stream’s source spec and VideoOp (JSON, zero fields omitted), k_enc and the nonce key HKDF-SHA256 expansions of XI_VIDEO_TRANSFORM_STREAM_KEYS (first seals, all open: rotation; empty derives them from the first signing key, else a random key per process with a startup warning), the nonce HMAC-SHA256(k_mac, expiry ‖ payload)[:12] (a synthetic IV: deterministic, so a manifest and its URLs are stable and cacheable, and two payloads never share a nonce). Properties: a ticket cannot be forged or altered (403), carries an expiry (XI_VIDEO_TRANSFORM_STREAM_TTL, default 12 h, capped by the URL’s own expires; past it 404 url expired, like an expired URL), hides the source (an enc/ source stays secret), has one spelling (strict base64), and opens only its own stream: the rendition and segment after it are checked against the stream’s layout (404 otherwise), and the source passes the allowlist again. The expiry moves in quarter-TTL steps so a manifest stays the same for a while; manifests are cacheable for a quarter of the TTL, parts until their ticket expires. Considered and rejected: an HMAC per child URL over (master key, rendition, segment) with the master’s query copied into every URL (the source and options would have to travel in clear, or be looked up), and a server-side session id (lost on restart and across instances without a shared store).
• Cache and prefetch. The probe, init segments, media segments and the audio track are keyed by the stream request’s canonical key, the source identity, the stream’s codecs and the part; they go to an in-memory stream cache (XI_VIDEO_TRANSFORM_STREAM_CACHE_BYTES, a cache/memory store) and the result cache (a persistent tier keeps them across instances and restarts). A part being transcoded is coalesced like any transcode. A segment transcoded for a request schedules the next XI_VIDEO_TRANSFORM_PREFETCH_SEGMENTS (default 2) of its rendition that the stream cache lacks, from the source the request fetched, through media.Service.Prefetch: it is admitted only when a run slot is free, nobody waits and (since WP-121c, after its probe) the segment’s cost fits in the free capacity at once, so a prefetch never queues and never delays a request (a request that joins a prefetch which then gives up runs the transcode itself); Service.Close stops the prefetches and waits for them at shutdown.
• Codecs. gpl: H.264 High; full: AV1 Main; vc:vp9 (VP9 in CMAF) and vc:av1 anywhere; AAC-LC audio in both. CMAF plays in HLS since version 7 and in DASH; MPEG-TS segments for older HLS players exist since WP-131 (H.264 only, so gpl only; “HLS extras”); HEVC is not streamed.
• Limits. The input limits apply as for files; the output duration limit does not (a stream is the whole cut; each segment is d long, each audio chunk XI_VIDEO_TRANSFORM_AUDIO_CHUNK long, each a transcode bounded by XI_VIDEO_TRANSFORM_TIMEOUT and _MAX_OUTPUT_BYTES). A track that cannot be chunked (coarse timestamps, loudnorm) is still one transcode and needs a timeout that covers it.
• CORS. XI_SERVER_CORS_ORIGINS (parsed since M1, applied only now) lets HLS and DASH players in another origin’s page load the parts.

7. Codecs and licences per flavour

	slim	full (LGPL FFmpeg)	gpl (GPL FFmpeg)
Video transforms	no (Unsupported, startup error with XI_VIDEO_TRANSFORM_ENABLED=true)	yes	yes
Video decode	none	H.264, HEVC, VP8, VP9, AV1 (dav1d), MPEG-4, MJPEG, PNG	same
Video encode	none	VP9, VP8 (libvpx, BSD), AV1 (libaom, BSD)	+ H.264 (libx264), HEVC (libx265)
Audio decode	none	AAC, MP3, MP2, Opus, Vorbis, FLAC, ALAC, AC-3, E-AC-3, PCM (native FFmpeg, LGPL)	same
Audio encode	none	AAC (native, LGPL), Opus (libopus, BSD-3-Clause), MP3 (LAME, LGPL-2.0+), FLAC, PCM	same
Containers	none	MP4/MOV/M4A, WebM/Matroska, MPEG-TS, HLS, DASH, MP3, Ogg/Opus, ADTS, WAV, FLAC	same

Defaults: mp4 = AV1 + AAC (full) or H.264 + AAC (gpl); webm = VP9 + Opus; m4a = AAC; mp3 = MP3; opus/ogg = Opus. FDK-AAC (nonfree) is never built (--enable-nonfree fails the licence gate). The base build, licence gate and checks are in docker/base/ (README “Full flavor”) and the ADR 0006 amendment of 2026-10-04. Patents: MP3’s have expired; AAC-LC, AC-3/E-AC-3 decode, H.264 and HEVC may be covered by pools in some jurisdictions; as for H.264/HEVC since 2026-09-27, the deployer reviews what applies.

8. Security and DoS posture

Video is the most expensive request a public endpoint can receive; the defaults assume hostile input.

• Signed only by default. XI_VIDEO_TRANSFORM_UNSIGNED_POLICY=deny (default): an unsigned video transformation answers 403 before anything is fetched, even on a mount whose unsigned policy is allow. allow leaves it to the mount’s policy, where restricted still refuses it (its width/quality allowlists describe images and do not admit KindVideo). restricted (WP-132, “Video presets” below) admits an unsigned video transformation only as its named video presets make it. Verified and trusted signatures pass.
• Input limits before decoding: XI_VIDEO_TRANSFORM_MAX_INPUT_BYTES (at fetch and again at open), _MAX_INPUT_DURATION and _MAX_INPUT_PIXELS (from the probe, before any decoder opens; the decoder’s max_pixels too), and demuxer/decoder whitelists (format_whitelist, codec_whitelist) per job; protocol_whitelist is empty (custom AVIO only), HLS/DASH/concat demuxers are not built, so a source cannot make FFmpeg open another file or URL.
• Output limits: XI_VIDEO_TRANSFORM_MAX_OUTPUT_DURATION (default 5 min), _MAX_OUTPUT_PIXELS (default 3840 × 2160), _MAX_FPS (60; a faster source is reduced to it, a faster request refused), the bit-rate range of transform.VideoOp (50 kbit/s to 200 Mbit/s video), _MAX_OUTPUT_BYTES (256 MiB, enforced by the muxer while writing) and _TIMEOUT (2 min wall clock). Duration, pixels and frame rate are checked by PlanTranscode from the probe before an encoder opens (422); output bytes and time while running. The defaults are wider than the first draft’s (60 s, 1080p) because each job is bounded in memory and time anyway; operators of public endpoints should lower them.
• Cost accounting (WP-121c, see “Cost-based admission” below). Each transcode is charged its estimated work (decode and encode pixels × frames × measured per-codec factors, audio seconds) against XI_VIDEO_TRANSFORM_CAPACITY, so ten 4K AV1 jobs do not get the same admission as ten 360p VP9 jobs; the probe now runs before admission under a small bound of its own. Metrics: xi_video_transform_duration_seconds{result}, _bytes_total{direction}, _coalesced_total, _queue_depth, _workers_busy, _overload_total{reason}, _job_cost, _capacity, _capacity_used, _queued_cost.
• Process isolation. FFmpeg runs in-process like libvips: whitelists, fuzzing of the shim’s entry points with the fixtures, and the video role keep a crash on the video fleet. If fuzzing or field reports show instability, the fallback is the subprocess sandbox of ADR 0011 (internal/sandbox) around a dedicated transcoder program; the job model (bytes in, bytes out; files once WP-121b spools) already fits it.

Alternatives considered

• A separate service (own repo or binary). Clean isolation, but it would duplicate sources and their credentials, signing and encryption, allowlists, the dialect parsers (Cloudinary and Cloudflare URLs mix image and video options), presets, cache stores, config schema, metrics and the release train, and every security fix would ship twice. The XI_ROLE split gives the same operational isolation (separate deployments, scaling and crash domains) from one codebase. Rejected.
• Go plugins (plugin package). Linux-only, the plugin and host must be built with the identical toolchain and module versions, plugins cannot be unloaded, and cgo inside a plugin adds nothing over a build tag; the flavour difference (FFmpeg or not) is already a build tag plus runtime encoder probing. Rejected.
• The ffmpeg CLI per job (exec.Command). Easiest way to get every filter, and a natural crash boundary. But a filter graph assembled as a command line is an injection surface that needs escaping rules for every filter, cancellation is a signal, progress and errors are parsed from stderr, the CLI re-probes what the server already probed, limits move to rlimits, and HLS JIT segments would pay process start-up per segment. ADR 0010 measured the CLI at 2–6× the shim for single frames. The shim with a built (not parsed) graph keeps typed parameters, the abort flag and in-process metrics. Kept only as the sandboxed fallback above, not the default. Rejected as the main path.
• libvips per frame (decode with FFmpeg, transform with libvips, encode with FFmpeg). Reuses image options literally, but costs two colour conversions and a copy per frame, has no audio, and would put libvips work inside a long-running encode. Rejected; libvips renders overlays only.
• GStreamer (LGPL). A second large multimedia framework next to FFmpeg, with plugins whose licences vary per element and more C on untrusted input. Rejected.
• go-astiav or another Go binding: rejected for the reasons in ADR 0010 (third-party cgo outside rule 3, exposes FFmpeg objects to Go).

Consequences

• The full and gpl base images grow by the audio encoders, libswresample and the extra muxers and filters (WP-120; sizes in docker/base/README.md); slim is unchanged.
• internal/video/ffmpeg grows a third C file; the leak, abort and fuzz tests of ADR 0010 extend to it.
• The config gains XI_ROLE and a video.transform section (XI_VIDEO_TRANSFORM_*: enabled, workers, queue, queue timeout, timeout, threads, input and output limits, unsigned policy), documented in docs/CONFIG.md by the WPs that add each setting.
• Work breakdown: ROADMAP M5 (WP-120 to WP-127, plus WP-121b spool files and WP-121c cost-based admission).

Reconciliation with WP-121 (2026-10-04)

WP-121 was implemented before this ADR was available. The integration branch (m5/foundation-integration) kept the implementation where it meets the ADR’s intent and changed the text above to match. Decisions:

1. Config names: XI_VIDEO_TRANSFORM_* (WP-121) instead of XI_VIDEO_WORKERS, XI_VIDEO_JOB_TIMEOUT, XI_VIDEO_ALLOW_UNSIGNED and so on, so transformation limits stay separate from the thumbnail XI_VIDEO_MAX_* keys.
2. Package: internal/media for the subsystem, the planner in internal/video and the model in transform.VideoOp, instead of internal/video/transcode and video.Transform.
3. Filter graph: chain text written by the planner from fixed names and numbers and parsed by avfilter_graph_parse_ptr, instead of one AVOption-configured node per filter. Kept because VideoOp has no string field and TestFilterChainsAreNumeric checks the alphabet; text options must use AVOption on filters the shim creates (§2).
4. Memory: source and output in memory for Phase A with the bound workers × (max input + max output); spool files are WP-121b.
5. Security defaults: XI_VIDEO_TRANSFORM_ENABLED false by default and fatal without FFmpeg (WP-121). The integration added the signed-only default (XI_VIDEO_TRANSFORM_UNSIGNED_POLICY=deny) and made the image role skip the pool, with tests. The input and output limits were already checked before an encoder opens. Cost-based admission is WP-121c.
6. Roles: as implemented: the image role answers video transformations 404, the video role answers everything but video transformations and health 404; §4 is aligned.
7. libswresample: not on the cgo pkg-config line, because the shim resamples only through the aresample filter; it is linked through libavfilter.

Dialects (WP-125, 2026-10-04)

The dialects map onto VideoOp as §1 intended; three fields were added, all numbers or enums: Gravity (where a cover crop keeps the frame and a pad places it), VideoCrop.Gravity (a crop anchored at a compass point, X/Y then offsets from it) and AutoContainer (Cloudinary f_auto, ImageKit f-auto: the server picks WebM when Accept names video/webm, else MP4, and answers Vary: Accept; the choice is made before the cache key, so each container has its own entry). Cloudflare mode=frame and mode=spritesheet reuse the thumbnail and tile paths (image requests, the image unsigned policy). Named presets that whitelist video transformations for unsigned URLs (§8) were not part of WP-125; WP-132 added them (“Video presets” below).

URL syntax and audio (WP-122, WP-123, 2026-10-05)

Decisions taken when the first URL syntax landed; the option tables are URL-DIALECTS §3.8–3.9.

1. A /video endpoint, not an output extension on processing URLs. imgproxy has no video transcoding to be compatible with, and @mp4 on an image URL already means GIF → MP4 (WP-76): the kind of a request cannot depend on what the source turns out to be after the fetch. So video transformations have their own grammar under <mount>/video (router.VideoParser, selected like /info): /video/%signature/%options/{plain/…@%container|%base64.%container|enc/…}, and /q/video?url=… in the query dialect. The prefix is also what ADR §4 wants for routing a video role: an operator sends /video/* (and /q/video) to the video deployment.
2. Domain-separated signatures. A /video signature covers "/video/" + the path after the signature (xi_sig covers "/video" + query), not the path alone as /info does: otherwise any signed image URL of a video source (a poster) would also be a signed transcode, the most expensive request there is, of the same options. @xtandard/image signs accordingly.
3. imgproxy’s names, video semantics. The geometry options keep imgproxy’s names and argument rules (rs, s, rt, w, h, el, ex, g, c, rot, fl, bg, q, f) and map onto the VideoFits (fit = scale-down, contain with el, pad with ex; fill and fill-down = cover; force = fill; auto refused); new options take descriptive names with short aliases (start/vs, end/ve, duration/vd, video_bitrate/vb, keyframe_interval/ki, sample_rate/sr, audio_channels/ach, volume/vol, loudnorm/ln, fade/fd, waveform/wf). Video options are their own namespace, like info options; no chained pipelines, and no presets until WP-132 added video presets (“Video presets” below; until then only-presets mode refused every /video URL). The dialect validates the VideoOp while parsing (VideoOp.Validate), so a bad value is a 400 naming the field before any fetch.
4. Model additions (all numbers, enums and colours; the chain alphabet test covers each): KeyframeInterval, FadeIn/FadeOut, SampleRate, Channels, Volume, Loudnorm (VideoLoudnorm: integrated, true peak, range within FFmpeg’s bounds; the target is per request, default EBU R128 -23 LUFS / -1 dBTP / 7 LU), Waveform (VideoWaveform: colour, background, split channels); containers ogg, flac, wav, png, webp; audio codecs flac and pcm; a VideoCrop side of 0 is the whole side. Gravity reuses WP-125’s VideoOp.Gravity and VideoCrop.Gravity (compass anchors): imgproxy’s g takes a compass point (its offsets move a crop that has no gravity of its own; fp and detection gravities are refused, since a focus point is not in the model and detection needs frames before the plan). loudnorm’s option names are written lower case (i, tp, lra) to stay inside the alphabet.
5. Waveforms through the image path. The shim runs the audio chain into showwavespic and a video buffer sink, encodes the one RGBA frame with the rawvideo encoder and muxer into the usual output buffer, and the engine (app) flattens it on an opaque background and saves PNG or lossless WebP with libvips inside vips.Run. FFmpeg’s PNG encoder would have covered PNG only; libvips gives WebP and the same encoders as images.
6. Audio sources. The shim’s demuxer whitelist gains the audio containers the base already builds (MP3, AAC/ADTS, AC-3, E-AC-3, Ogg, FLAC, WAV) and the decoder whitelist their codecs, so audio files are sources of audio outputs and waveforms. /info still answers 422 for an audio-only file (it describes images and videos); its video streams gain the channel count.
7. Overlays in the URL (with WP-124): imgproxy’s watermark options (wm, wmu, wmt, wms, wmr, wmsh) are parsed exactly as for images into the pipeline’s watermark and then moved into VideoOp.Overlays (one image or text overlay, drawn only with an opacity above 0, as on images), timed by the video-only overlay_window/ow and overlay_fade/of; timecode/tc adds the running timecode overlay. The /_video test router of WP-121/124 is gone: the app tests use signed /video URLs.
8. Not done: keyframe-only (“fast”) cuts (exact cuts only: the ADR §2 option needs the keyframe time before planning the audio offset) and /info reporting the planned output (it reports the source).

Cost-based admission (WP-121c, 2026-10-06)

Until WP-121c every job took one worker slot, so a 5-minute 4K AV1 encode and a 2 s audio extraction weighed the same: a few heavy jobs held the pool, or cheap ones waited behind them.

• Cost model (video.EstimateCost, pure, table-tested). One unit is about a millisecond of one core’s work on the reference build (the gpl base builder, one codec thread, the default speeds and presets): 2 + decoded s × source fps × source pixels × decode(codec) + output s × output fps × output pixels × encode(codec) + audio, where a cut that does not start at the beginning decodes one second of pre-roll and a cut with an end one second of tail (the shim’s XI_TX_END_SLACK), audio is base + seconds × per-second(codec) × channels/2 × rate/48 kHz, and a waveform seconds × 1.1 + megapixels × 2. Audio-only outputs and waveforms decode no video; HLS/DASH segments are their window (each is its own job), an audio track the whole stream. The factors are fitted to the medians of BenchmarkTranscodeCost (internal/video/ffmpeg/testdata/bench-transcode-cost-fit.txt, -count=10, gpl builder; bench-transcode-cost.txt is a later run of the final model reporting est-ms and est/measured per case), measured, not guessed:

  Encoder (default settings)	ns per output pixel and frame	Decoder	ns per source pixel and frame	Audio encoder	ms per second (stereo 48 kHz)
  HEVC (x265 veryfast)	35	hevc	2.2	AAC (FFmpeg)	20.7 (+18 start-up)
  AV1 (libaom speed 8)	21	av1	2.4	MP3 (LAME)	7.9
  H.264 (x264 veryfast)	8	h264, vp8	2.0	Opus	7.0
  VP9 (libvpx speed 8)	7.5	vp9	2.0	FLAC / PCM	2.2 / 1.0
  		others	3.0

  So the order is HEVC > AV1 > H.264 ≈ VP9, not the AV1 > VP9 ≈ HEVC > H.264 first assumed: at their default settings libaom and libvpx run their fast realtime modes and x265 veryfast is the slowest. On real footage (Big Buck Bunny, 180p to 1080p outputs) the estimate is 0.78–0.96 of the measured time for every encoder; on the procedural fixtures, which decode and encode much faster, up to 1.93 (2 s segments), so every fixture is within 2×. The check run, on a busier host, gave 0.52–1.75 except x265 at 360p (0.43), which a quiet re-run measured at 0.79–0.89. Content complexity, non-default speeds and presets, overlays and multi-threading are not modelled; the capacity is a budget of work, not a CPU model.

• Probe before admission. A job holds a place in the pool from its arrival (MAX_JOBS + QUEUE places, TryAcquire: 429 when full), then probes (Engine.Open: the container’s headers and FFmpeg’s stream analysis, which decodes a few packets, never the cut) and plans under a probe bound of WORKERS at a time, then waits for admission. The probe and the wait share XI_VIDEO_TRANSFORM_QUEUE_TIMEOUT (503). The opened job is kept, not probed twice.

• Weighted admission. Two semaphore.Weighted: run slots (XI_VIDEO_TRANSFORM_MAX_JOBS, default 2 × WORKERS, so cheap jobs can run beside heavy ones) and cost units (XI_VIDEO_TRANSFORM_CAPACITY, default WORKERS × 10000: ten seconds of work per worker, so a pool of reference-sized jobs behaves as before). A job that cannot start at once queues unless the cost already queued plus its own is over XI_VIDEO_TRANSFORM_QUEUE_COST (default max(QUEUE, 1) × 10000; 429 queue_cost); a job always queues behind an empty queue. A cost above the capacity is clamped to it: the job runs alone instead of being refused (the first ROADMAP row said 422; the WP-121c brief chose clamping, since the output limits already bound the worst job and a 422 would refuse work the pool can do). The estimate is observed before clamping.

• Fairness. semaphore.Weighted is FIFO (x/sync v0.23.0: Acquire queues behind earlier waiters, TryAcquire fails while anyone waits, notifyWaiters wakes from the front and stops at the first waiter that does not fit, “to avoid starvation of large requests”). A large job at the head is never overtaken, so it cannot starve; the price is head-of-line blocking, small jobs arriving after it wait for it to start. Tested with synctest (TestAdmissionInterleave).

• Prefetch. A prefetch takes a place only when nobody is queued and a run slot and some capacity are free, probes only if a probe slot is free, and starts only if the segment’s cost fits at once (TryAcquire); otherwise it gives up and holds nothing. A request that joined it starts the transcode itself.

• Memory. Up to MAX_JOBS outputs and MAX_JOBS + QUEUE sources (with their opened demuxers) are held at once; with WP-121b (spool files, next section) each of them holds at most XI_VIDEO_TRANSFORM_SPOOL_THRESHOLD in memory, the rest on disk.

• Retry-After keeps the image limiter’s estimate (average job time per round of queued jobs).

Spool files (WP-121b, 2026-10-06)

Large video sources and outputs live in temporary files instead of memory, so a transcode’s memory no longer grows with MAX_INPUT_BYTES + MAX_OUTPUT_BYTES (CONFIG.md XI_VIDEO_TRANSFORM_SPOOL_*, RELEASE.md “Running read-only”).

1. internal/spool (pure Go) owns the directory (XI_VIDEO_TRANSFORM_SPOOL_DIR, default $TMPDIR/xtandard-video): created 0700, checked writable at startup (an instance that serves video transformations fails to start otherwise), cleared of xi-spool-* files of an earlier process (nothing else is touched). Create opens a file with os.CreateTemp (0600) and unlinks it at once: the descriptor is its only name, so it disappears on every path (success, error, cancellation, a panic, a crash) without cleanup code, and the directory itself stays empty. A Writer fills it (from Go, or from C through the descriptor) and Finish turns it into a *cache.FileBuffer, a refcounted Buffer read with ReadAt (and an io.ReadSeeker for http.ServeContent) whose last Release closes the file.
2. Budget. XI_VIDEO_TRANSFORM_SPOOL_MAX_BYTES (0: none) is a semaphore.Weighted of bytes, taken with TryAcquire only (fail fast): a download reserves its declared length (or grows its reservation in 4 MiB steps while it reads a body of unknown length), an encode reserves XI_VIDEO_TRANSFORM_MAX_OUTPUT_BYTES while it runs (C cannot ask Go for more) and shrinks to the file’s size at the end; the last Release returns it. A reservation that does not fit first asks the fetcher to drop the spooled sources it keeps, then fails with *spool.BudgetError inside an errs.KindUnavailable: 503, and Retry-After: 5 from server.writeError (the one mapping place, through an interface, so internal/limiter is untouched). xi_video_spool_bytes, _files, _spills_total{kind}, _rejections_total.
3. How FFmpeg reads a spooled source: a descriptor, not a path or a callback. Three ways were considered. (a) A path opened by the shim through FFmpeg’s file protocol would put a protocol back into protocol_whitelist (empty since WP-121), and with it a way for a demuxer to open other files. (b) Read/seek callbacks into Go through runtime/cgo.Handle keep the whitelist empty but make every 64 KiB read a C→Go call on the transcoder’s thread, and would be the shim’s first callback into Go (the nocallback annotations would go). (c) Chosen: the existing read/seek AVIO of the shim reads with pread on a duplicate of the file’s descriptor (fcntl(F_DUPFD_CLOEXEC), so a subprocess such as the EPS sandbox never inherits it) that the handle owns and closes (xi_tx_new_fd, which also takes an offset). No protocol, no path, no Go pointer and no callback crosses into C: the whitelist and the nocallback/noescape annotations stay as they were, and pread never moves a shared file offset. Probes and outputs of a file source are identical to those of the same bytes in memory (TestTranscodeFromFile).
4. Outputs spill. The output starts in the growable C buffer; XiTxParams.spill_fd/spill_bytes give the shim a spool file (a duplicated descriptor again) and the threshold, and the first write that would grow the output past it copies the buffer into the file, frees it and continues with pwrite/pread at the output’s position: seeks, the faststart rewrite and cues_to_front read back and rewrite the file through the same io_open hooks, and the limit of XI_VIDEO_TRANSFORM_MAX_OUTPUT_BYTES still applies (TestTranscodeSpill: MP4, WebM and M4A spilled byte-identical to memory). The engine (app) prepares the file only for whole outputs whose limit passes the threshold: stream parts (cut by media/fmp4 in memory) and waveform pixels stay in memory. An output that stays small is returned from memory and the file discarded.
5. Serving and caching. respondVideo serves cache.BodyReader(body): a file body is read range by range, never mapped (tested). cache.FileBuffer.Bytes still works for the rare caller that needs a slice (it maps the file read-only, page cache rather than heap). Cache tiers: the memory tier skips file bodies (holding one would pin the file and its budget); the disk tier streams them in (CRC-32C computed while streaming) and answers a hit of at least disk.DefaultFileBodyBytes (8 MiB) with a FileBuffer over the entry file after reading only its frame (length checked against the frame; the CRC is not, to keep a Range hit from reading the whole file: a torn file shows as a length mismatch); redis checks its value limit on the length; S3 uploads from the file. Deviation from the ROADMAP row: S3 hits are still read whole (no ranged GET); put a disk tier in front, which Tiered fills from S3.
6. Sharing sources. fetch.Fetcher.FetchLarge (video requests and every stream part) spools a source above the threshold and keeps it for XI_VIDEO_TRANSFORM_SPOOL_TTL (default 1 min): reused without the origin for that long after its download, kept while it is used at least once per period (then revalidated with If-None-Match/If-Modified-Since, 304 keeps the same file), dropped after a period without use (time.AfterFunc, no goroutine) or when the budget is full; concurrent calls share one download (a flight of its own, so an image Fetch of the same key never receives a file). The cache is keyed by the source key (source, bucket, path, version) and validated by the origin’s ETag/Last-Modified. Measured: the manifest and every segment of an HLS stream took 13 downloads of the source without the spool (origin cache off) and 1 with it (TestStreamSharesSpooledSource).
7. Measured (BenchmarkVideoSpool, a 104 MB uncompressed AVI generated by the test, 640×360 VP9 output of 3.2 MB, origin and result caches off, each variant in its own process, gpl builder on arm64, -count=10 by process, internal/app/testdata/bench-spool.txt): peak RSS 399 → 301 MiB (-25 %), RSS growth during the transcodes 267 → 168 MiB (-37 %) (the difference is the source; the rest is the decoder, the encoder and the client), Go B/op unchanged (the source was never on the Go heap); wall time unchanged (671 vs 695 ms per transcode, p = 0.32).

Video presets (WP-132, 2026-10-15)

Operators wanted to put video on public pages without signing every URL and without opening every option (§8). imgproxy’s answer for images is presets with IMGPROXY_ONLY_PRESETS; Cloudinary’s and ImageKit’s are named transformations (t_, n-) plus “strict transformations”, which refuse anything an account has not allowed. WP-132 does the same for video transformations.

1. Video presets are a store of their own. XI_VIDEO_TRANSFORM_PRESETS (name=options,…, YAML video.transform.presets) holds /video options (URL-DIALECTS §3.8) in imgproxy’s preset syntax, parsed by the same internal/preset package as the image XI_PRESETS, which stay image options: the two option tables differ (w is a video size but bl exists only for images), so one name cannot mean both. A video preset has one pipeline and no delivery or source options (cb, exp, fn, att, hs, bc, ct): it describes a transformation, not a response. The imgproxy dialect expands pr: on /video in place with imgproxy’s rules (later options win, nesting, default first; XI_PRESETS_ONLY admits /video URLs made only of video preset references). /q/video takes preset= through the same builder. Cloudinary (t_) and ImageKit (n-) build their VideoOp with their own state machines, so for them a preset is the op’s starting point: imgproxy.Parser.VideoPresetOp gives the presets’ finished op (after default), the dialect applies its parameters on top, and a preset that chose a container or a stream keeps it unless f_/f-/ik-video.mp4 say otherwise (a preset’s stream fills an ik-master stream’s renditions and segment length unless sr- sets them). Cloudinary also documents a named transformation as the defaults of its component, which this matches for video’s single accumulating op.
2. Startup validation. Config.Validate checks names and syntax (video.transform.presets (XI_VIDEO_TRANSFORM_PRESETS)), without importing the dialects. Building the dialects applies every preset to an empty /video request (an unknown option, a bad value, a chained pipeline or a delivery option fails startup) and compares the result with the output limits (_MAX_OUTPUT_PIXELS, _MAX_FPS, _MAX_OUTPUT_DURATION): a preset that every request would exceed is a configuration error, not a 422 per request. Costs that depend on the source (duration, input size) cannot be checked before a probe and stay with cost-based admission.
3. The restricted policy compares ops, not option names. Listing allowed option names per dialect would need a table per dialect and would miss options added later. Instead, for an unsigned video request that names a video preset, the dialect parses the URL a second time keeping only its preset references (no other option, layer, format, extension, ik-master or ik-video.mp4), and returns that op in dialect.Result.PresetVideo. The router (router.VideoPolicy, the one place that sees the mount’s policy too) admits the request when the two ops are equal apart from the free options of XI_VIDEO_TRANSFORM_UNSIGNED_ALLOWED_OPTIONS (start, end, duration, mute, by meaning, so vs:, so_ and so- are the same option) and a container of XI_VIDEO_TRANSFORM_UNSIGNED_FORMATS of the same kind (video, audio only, waveform; an automatic WebM-or-MP4 choice needs both). The comparison is reflect.DeepEqual over the whole VideoOp, so a field added to it later fails closed until it is listed. Consequences: an option the preset overrides anyway (w:1920/pr:web) or that repeats the preset’s value passes, because the output is the preset’s; HLS/DASH and raw:1 are possible only when a preset asks for them, since the extension and ik-master are left out of the presets-only parse; the request’s other parts follow the image restricted policy (no security options, chained pipelines, cache busters or cache bypass; filename, attachment and expiry allowed). When the free options change the cut and XI_VIDEO_TRANSFORM_UNSIGNED_MAX_DURATION is set, the cut must have a duration or an end and fit in it, decided without a probe.
4. Policy precedence. A mount whose policy is deny refuses every unsigned request, video included; otherwise the video policy decides: deny refuses, allow defers to the mount (allow passes, restricted refuses, as before), restricted applies point 3 on allow and restricted mounts alike, so XI_SECURITY_UNSIGNED_POLICY=restricted (the default) and public preset video coexist. Refusals are the router’s 403 (unsigned request not allowed, 404 with XI_COMPAT_IMGPROXY_STATUS), before anything is fetched; the server keeps its own deny check as a second line. Signed and trusted URLs are never compared.
5. Rejected: requiring presets alone with no free options (players need start for previews and chapters; the free set is empty by default); letting the extension choose a stream (a public URL could turn any preset into a ladder of transcodes); applying the image presets to videos (different option tables); a per-dialect allowlist of option names (point 3).

HLS extras (WP-131, 2026-10-15)

Three things WP-126 left out of §6, user-facing semantics in URL-DIALECTS §3.10 and FORMATS §5.2.

1. MPEG-TS segments. HLS players older than protocol version 7 (smart TVs, set-top boxes, early Android) play only MPEG-TS. A stream asks for them with VideoStream.Format (transform.SegmentTS): imgproxy and query segment_format:ts (sgf; fmp4 is the default), Cloudinary fl_hlsv3 on an .m3u8 output (Cloudinary’s flag for HLS version 3 with .ts segments; recorded as not applied on .mpd and files). ImageKit has no equivalent; Cloudflare Stream is another product.
   • Codec. HLS maps H.264 and HEVC into MPEG-TS, not AV1 or VP9, and HEVC is not streamed, so TS streams are H.264 High + AAC-LC: gpl writes them, full (no H.264 encoder: libx264 is GPL and openh264 was ruled out) answers 422 from ResolveStream before anything is fetched; vc:vp9/vc:av1 with TS, and TS with DASH, are 400.
   • Muxed segments. Each v<i>/<n>.ts carries the video of its segment and the audio of the same span (one EXT-X-STREAM-INF per rendition with both codecs, version 3 playlists, no EXT-X-MAP, no audio group), since a separate audio rendition in TS (or packed audio with ID3 timestamps) is what the oldest players handle worst. The audio is not encoded per segment and rendition: the segment’s fragment of the stream’s CMAF audio track (the same chunks as for CMAF, cached) is parsed (fmp4.File.Samples) and its AAC packets copied into the TS by decode time (XiTxParams.copy_*, C memory for the call; the muxer writes ADTS from a fixed AudioSpecificConfig: AAC-LC, 48 kHz, the channel count). Every rendition and both formats play the same audio, encoded once.
   • Independent segments. A segment is muxed on its own (mpegts): PAT and PMT before its elementary-stream packets, an IDR frame first (the encoder of each segment starts with one; SPS/PPS in band, AUDs from the muxer), timestamps the stream’s (90 kHz, the same stream offset as CMAF, mpegts_copyts, no muxing delay), so the joined segments form one timeline. Continuity counters cannot run on across segments encoded independently (a segment would need the packet counts of every segment before it), so the first packet of every PID has the discontinuity indicator (mpegts_flags=+initial_discontinuity), which makes the counter jump legal (ISO/IEC 13818-1 §2.4.3.5); the PCR values themselves stay continuous. Tested: the joined segments of both renditions have the source’s 165 frames on the 25 fps grid, a keyframe per segment, gapless audio over the stream in sync with the first frame, counters that only jump at discontinuity indicators, PAT/PMT before the first PES packet of every segment, and FFmpeg decodes the joined stream without a warning.
2. Overlays on streams. Image and text watermarks and timecodes (WP-124) are drawn on every video segment of every rendition, CMAF and TS. Timing: the frames of a segment enter the overlay graph with the stream’s timestamps (§6: 0 is the stream’s start), so windows, fades and the timecode are on the stream’s timeline, as on a file of the same cut, across segment boundaries (tested with a window that spans two boundaries, in exactly the frames inside it). Scale: an overlay is laid out on the largest rendition (as on a file of its size) and scaled to the others: the rendered RGBA (exec.RenderOverlayScaled: the rectangle scaled and rounded inside the frame, the pixels resized premultiplied with Lanczos 3), a timecode’s font size, offsets and shadow (video.planOverlaysOn), so an ABR switch never moves or resizes it. Laying out per rendition was rejected (an absolute-pixel watermark, imgproxy wms:0 or Cloudinary w_120, would cover a larger part of the small renditions and jump at every switch), and the source’s size as the reference too (a 4K source with a 1080p top rung would shrink pixel offsets and sizes for no visible reason). Plumbing: the overlays travel in the ticket like every other option; the manifest checks their sources against the allowlist before a ticket names them; a segment request fetches the overlay images once (the watermark path: allowlist, fetcher, byte limit) and hands them to its transcode and its prefetches. The graph invariants are unchanged (TestOverlayGraphIsNumeric covers every segment of a three-rendition stream).
3. Chunked stream audio. The track is encoded in chunks of whole segments (XI_VIDEO_TRANSFORM_AUDIO_CHUNK, default 30 s; 0 is the whole track), each its own transcode admitted by its own cost (video.EstimateCost of the chunk’s window): a request for an audio segment transcodes its chunk, caches every part of it (the init segment comes with each chunk and is byte-identical), and prefetches the next chunk. The chunks join as one encode would (video.PlanAudioChunk):
   • One packet grid. Packet j of the track covers output samples [g0 + (j−1)·1024, g0 + j·1024) (48 kHz, 0 at the stream’s start; packet 0 is the priming; g0 the first audio sample’s place when the audio starts after the video). A chunk’s encoder input starts on that grid (an atrim on the 48 kHz timeline), and FFmpeg’s AAC encoder stamps each packet with the time of its first input sample minus the priming, so packet j covers the same samples in every chunk, and fragment boundaries (the first packet at or after k·d) are those of a single encode.
   • Pre-roll and post-roll. The input starts 16 packets (≈ 0.34 s) before the chunk’s first packet, so the overlapping MDCT windows, the encoder’s psychoacoustic and rate state and the source decoder are warm, and ends 4 packets after its last (the encoder looks ahead one frame for its window shapes); the shim drops the packets outside the chunk’s segments (frag_first, frag_end). The final packet of the stream goes into the last segment when none reached it (frag_final: a last segment shorter than a packet would otherwise have no audio; this also fixes that edge of the WP-126 single encode).
   • Resampler phase. A source not at 48 kHz goes through aresample, whose output instants depend on its first input sample: every chunk starts it on a source sample whose distance from the track’s first sample is a multiple of the rate ratio’s numerator (147 for 44.1 kHz → 48 kHz, whose 160 output samples then land exactly), a margin before the samples it keeps, with integer trims and an integer asetpts in source ticks (no rounding of a sample’s time).
   • Exact timestamps only. This needs audio timestamps that name samples exactly: a time base of 1/(k·rate) (MP4, MOV, Ogg, FLAC, WAV, MP3; the probe now reports the audio time base and start). Matroska and WebM count milliseconds, so a chunk could not place its first sample to better than ±24 samples and the joins would click; their tracks, and streams with loudnorm (a gain that adapts over the whole track), stay one transcode, as in WP-126 (and need a timeout that covers them).
   • A continuous sample clock. The shim now lets a stream’s audio frame that starts within 100 ms of the previous one’s end continue it (XI_TX_SNAP): coarse source timestamps (the WebM fixture’s Opus pre-skip rounded to a millisecond) left a 24-sample timestamp jump after the first packet, which the TS segments showed as a gap; a larger gap is the source’s and stays.
   • Measured (TestStreamAudioChunks, the new fixture stream-vp9-aac44k.mp4: 44.1 kHz AAC, 440/660 Hz tones; 1 s segments in 2 s chunks, against the single encode of the same layout): identical fragment boundaries, durations and init segments; the same decoded sample count; 37.4 dB from the single encode overall; around the joins (±1024 samples at 2, 4 and 6 s) the difference is smaller than elsewhere (rms 12.6 vs 31.0 of 16-bit full scale) and no click (largest second difference 892 there, 1342 elsewhere). End to end (TestStreamAudioChunksEndToEnd), the HLS audio segments fetched out of order and the MPEG-TS segments built from the chunks are gapless over the stream’s length.
4. Not done. Continuous continuity counters across TS segments (see 1), MPEG-TS for DASH, TS for AV1/VP9 (no HLS mapping), I-frame playlists, chunking for Matroska/WebM sources (they would need sample-exact positions from the decoder, e.g. counting samples from a known packet) and for loudnorm (a two-pass measurement first).