Skip to main content

The video pipeline

Everything between "a file arrives" and "a viewer presses play" is one queue-driven pipeline. It is asynchronous by design: the upload finishes long before the encode does, and the video becomes playable when the encode lands.

Video on demand

Ingest

Three ways in: a direct upload, a chunked resumable upload, and an async URL import (SSRF-guarded, with an optional sandboxed yt-dlp extractor behind the ytdlp profile). Per-user storage quotas apply to all three. Channel auto-sync can mirror an external channel's uploads on a schedule.

ClamAV scanning is optional — the scan profile plus MALWARE_SCAN_ENABLED=true. Budget an extra 2 GB of RAM for it; clamd loads the whole signature database into memory.

Encode and package

Encoding and packaging are one ffmpeg invocation, not two passes: the muxer is an output of the same run that drives the encoder.

  • Ladder: fixed-bitrate H.264 with AAC audio. Ladder planning is input-aware — it never upscales, keeps a native fallback rung, and caps frame rate.
  • Packaging: TRANSCODING_PACKAGER=cmaf is the default and writes one set of CMAF/fMP4 segments addressed by both an MPEG-DASH manifest and HLS playlists. ts is the legacy MPEG-TS-plus-HLS packager, kept as a config-only rollback: it affects new transcodes only, every video records the format its own tree was written in, and there is deliberately no back-catalogue re-packaging job.
  • Trick-play: an I-frame rendition per rung, which is what makes scrub previews work.
  • Also produced: progressive web videos, an optional VP9/WebM download alternate, thumbnails, storyboards and chapters.
  • HEVC and AV1 are behind TRANSCODING_HEVC_ENABLED and TRANSCODING_AV1_ENABLED, both default off, both CMAF-only. Each adds a further encoding of every ladder rung.
  • Hardware transcode is behind TRANSCODING_HWoff by default, with videotoolbox, vaapi, qsv and nvenc available. It requires the CMAF packager; the MPEG-TS path stays software H.264. AV1 is never hardware-encoded whatever the setting says.
What a job actually costs

One TargetAll job on a 1080p source is twelve full-source encode passes — four ladder rungs, four trick-play I-frame encodes and four progressive web videos — and runs at roughly 1.5–2.5× the source duration on 4 vCPU at -preset veryfast. This is why the host is sized for transcoding; see Requirements and sizing.

Captions are optional and asynchronous: with the captions profile on, the audio is handed to the Whisper service and the resulting WebVTT is stored alongside the video. Uploaded WebVTT works with or without it.

Deliver

Every media byte is proxied through the Go API, with a per-request database-backed authorisation check — the API is the visibility gate for originals, HLS and thumbnails. Playlists are rewritten in flight: a ?pt= HMAC token propagates to child requests for password-protected videos, and a ?v= generation key makes a completed transcode's URLs immutably cacheable.

Cache policy is emitted per asset shape, which is why a blanket public rule on /api/v1/videos/* is wrong — a video can become private or be deleted, and playback tokens appear in the query string:

AssetPolicy
Versioned VOD HLS (?v=<generation>)private, max-age=31536000, immutable
Unversioned VOD HLS compatibility URLprivate, max-age=0, must-revalidate
Authenticated or ?pt= mediaprivate, no-store
Live playlistno-cache, no-store
Live segmentprivate, max-age=12
Replaceable thumbnailprivate, max-age=300, must-revalidate

The original-file route supports Accept-Ranges and 206 Partial Content on both local and S3 storage, so any CDN or proxy in front must preserve Range requests — that is what seeking is.

Live

Live runs behind the media compose profile. RTMP ingest publishes 0.0.0.0:1935 around the edge — OBS on a creator's laptop dials it directly and a reverse proxy cannot stand in front of it, so 1935 is one of the two ports that must be open in your firewall when the profile is on.

The live HLS output goes through the same privacy gate as VOD, and a finished stream can be replayed to VOD, at which point it becomes an ordinary transcode job.

LIVE_INGEST_SECRET signs stream keys. Rotating it is free: it breaks any in-flight session, and new stream keys work immediately.

What is not in the pipeline

  • No in-player peer-to-peer. Viewer-to-viewer distribution is deferred by decision. Offload happens at the CDN and IPFS layers instead; see Storage and media.
  • No EME or DRM in the player.