FFmpeg filter_complex patterns — branching, merging, and multi-output graphs

ffmpeg's filter_complex is the powerful but cryptic tool that handles everything beyond linear filter chains — multi-input processing, branching graphs, multi-output pipelines, audio/video joint processing. Mastering filter_complex unlocks single-pass production pipelines that would otherwise require multiple ffmpeg invocations and intermediate files. This page is the engineering reference for practical filter_complex patterns.

#What filter_complex is

filter_complex enables filter graphs more sophisticated than the linear chain -vf and -af provide:

• Multiple inputs — combine multiple source files in one pipeline.
• Multiple outputs — produce several derived outputs from one pipeline.
• Branching — split a stream into multiple paths.
• Merging — combine multiple streams into one output.
• Cross-stream processing — operate on audio and video together.

The syntax uses square brackets to label streams between filters:

-filter_complex "[0:v]split=2[v1][v2]; [v1]scale=1920:1080[full]; [v2]scale=320:180[thumb]"

Three filters in sequence: split produces two outputs labeled [v1] and [v2]; each is then scaled separately.

#The labeling system

Square brackets identify streams:

• [INDEX:STREAM_TYPE] — input streams. [0:v] is video stream of input 0; [1:a] is audio of input 1.
• [NAME] — intermediate streams within the graph. [v1], [full], [mixed], etc. — any name you choose.
• [INDEX:STREAM] — alternative input notation. [0:0] is stream 0 of input 0.

Streams flow between filters. Each filter takes labeled inputs and produces labeled outputs.

The graph is described as a series of filter operations separated by semicolons:

-filter_complex "
  [0:v]filter1[temp1];
  [temp1]filter2[temp2];
  [temp2]filter3[output]
"

Each line declares a filter, its inputs, and its outputs.

#Common patterns

Pattern 1: Split video for multiple outputs

ffmpeg -i input.mp4 -filter_complex \
  "[0:v]split=2[full][small]; \
   [full]scale=1920:1080[v_full]; \
   [small]scale=640:360[v_small]" \
  -map "[v_full]" -c:v libx264 -crf 22 full.mp4 \
  -map "[v_small]" -c:v libx264 -crf 22 small.mp4

The split filter creates two copies; each goes to a different scaling filter; each is mapped to a separate output. One ffmpeg invocation produces two different-sized files.

Pattern 2: Picture-in-picture overlay

ffmpeg -i main.mp4 -i pip.mp4 -filter_complex \
  "[0:v][1:v]overlay=W-w-10:H-h-10[output]" \
  -map "[output]" -c:v libx264 -crf 22 output.mp4

Overlays input 1 (PiP) onto input 0 (main video) with 10-pixel margin from bottom-right.

Pattern 3: Side-by-side comparison

ffmpeg -i source.mp4 -i encoded.mp4 -filter_complex \
  "[0:v]scale=960:540[left]; \
   [1:v]scale=960:540[right]; \
   [left][right]hstack[output]" \
  -map "[output]" -c:v libx264 -crf 18 comparison.mp4

Useful for visual quality comparison — encode A on left, encode B on right, scrub to identify differences.

Pattern 4: Audio mixing

ffmpeg -i background_music.mp3 -i voice.wav -filter_complex \
  "[0:a]volume=0.3[bg]; \
   [1:a]volume=1.0[voice]; \
   [bg][voice]amix=inputs=2:duration=longest[mixed]" \
  -map "[mixed]" -c:a aac mixed.aac

Reduces background music volume; keeps voice at full; mixes them. Common pattern for narrated content.

Pattern 5: Watermark/logo overlay with timing

ffmpeg -i video.mp4 -i logo.png -filter_complex \
  "[1:v]scale=200:80[scaled_logo]; \
   [0:v][scaled_logo]overlay=10:10:enable='between(t,0,5)+gte(t,30)'[output]" \
  -map "[output]" -map 0:a -c:v libx264 -c:a copy output.mp4

Overlays scaled logo at top-left, visible from 0-5 seconds and after 30 seconds. The enable expression controls when the overlay appears.

Pattern 6: Multiple ABR variants in one pass

ffmpeg -i master.mp4 -filter_complex \
  "[0:v]split=4[v1][v2][v3][v4]; \
   [v1]scale=1920:1080[v_1080]; \
   [v2]scale=1280:720[v_720]; \
   [v3]scale=854:480[v_480]; \
   [v4]scale=640:360[v_360]" \
  -map "[v_1080]" -c:v libx264 -crf 22 -b:v 4M out_1080.mp4 \
  -map "[v_720]" -c:v libx264 -crf 22 -b:v 2.5M out_720.mp4 \
  -map "[v_480]" -c:v libx264 -crf 22 -b:v 1.5M out_480.mp4 \
  -map "[v_360]" -c:v libx264 -crf 22 -b:v 800k out_360.mp4

One source decode, four parallel scales, four parallel encodes. Significantly faster than four separate ffmpeg invocations because the source is decoded once.

Pattern 7: Concatenate clips

ffmpeg -i clip1.mp4 -i clip2.mp4 -i clip3.mp4 -filter_complex \
  "[0:v][0:a][1:v][1:a][2:v][2:a]concat=n=3:v=1:a=1[outv][outa]" \
  -map "[outv]" -map "[outa]" -c:v libx264 -c:a aac concatenated.mp4

Concatenates three clips with their respective audio. Useful for editorial assembly.

#Audio + video joint processing

filter_complex handles audio and video together when needed:

-filter_complex \
  "[0:v]scale=1920:1080[v]; \
   [0:a]aresample=48000:async=1[a]" \
  -map "[v]" -map "[a]"

Scales video; resamples audio to 48 kHz with sync correction. Both go to output.

For more sophisticated audio processing (compression, EQ, normalization) alongside video:

-filter_complex \
  "[0:v]scale=1920:1080,format=yuv420p[v]; \
   [0:a]highpass=f=80,acompressor,loudnorm[a]"

Removes low frequencies, applies compression, then loudness normalization.

#Performance considerations

filter_complex performance varies:

• Linear chains (single filter sequence) are typically fastest.
• Splits + parallel processing parallelize well across CPU cores.
• Multi-output encodes can saturate the source decoder if it's the bottleneck.
• Cross-stream operations (concat, mix) are typically fast.

For pipelines with heavy filter graphs, profile to find bottlenecks. Sometimes it's worth splitting one ffmpeg invocation into multiple sequential invocations with intermediate files; sometimes the parallelism wins.

#Hardware acceleration with filter_complex

Hardware-accelerated filters (scale_cuda, scale_qsv, scale_vaapi) work in filter_complex but require the data to stay on GPU:

ffmpeg -hwaccel cuda -hwaccel_output_format cuda \
  -i input.mp4 \
  -filter_complex \
  "[0:v]split=2[v1][v2]; \
   [v1]scale_cuda=1920:1080[v_full]; \
   [v2]scale_cuda=640:360[v_small]" \
  -map "[v_full]" -c:v h264_nvenc -preset p5 full.mp4 \
  -map "[v_small]" -c:v h264_nvenc -preset p5 small.mp4

The split happens on GPU; both scales happen on GPU; both encodes happen on GPU. Data never leaves GPU memory.

For software filters mixed with hardware, transitions between GPU and CPU memory cost throughput. Either keep everything on GPU or everything on CPU; mixed pipelines are slow.

#Debugging filter_complex

Filter graphs can be cryptic. Debugging tips:

Verbose logging:

ffmpeg -loglevel verbose -i input.mp4 -filter_complex "..." ...

Verbose output shows filter graph construction and processing details.

Filter graph dump:

ffmpeg -i input.mp4 -filter_complex "..." -filter_complex_threads 1 -dump output.mp4

Some ffmpeg versions support graph dumping; check your version.

Smaller test cases:

When a complex graph fails, simplify until it works, then add back complexity. Isolate the failing filter.

Filter help:

ffmpeg -h filter=overlay

Shows filter-specific help with options and examples.

#Common filter_complex bugs

Bug 1: Stream index errors.

[0:v] references the first video stream of the first input. If the input has no video stream, this fails. Verify input streams with ffprobe before constructing the graph.

Bug 2: Unmatched labels.

A filter outputs [temp] but no later filter uses [temp]. Or a filter expects [temp] but no earlier filter produces it. Match labels carefully.

Bug 3: Missing semicolons.

Filter declarations are separated by ;. Forget one and the parser misinterprets.

Bug 4: Memory location mismatch.

Mixing CPU and GPU filters without explicit hwupload/hwdownload causes errors or silent CPU fallback.

Bug 5: Output format compatibility.

Some filters output specific pixel formats; downstream filters may need conversion. Add format=yuv420p (or appropriate) where needed.

#Operational considerations

Things that matter for filter_complex in production:

• Test on representative content — graph behavior may differ for unusual content (very tall vs wide videos, unusual frame rates).
• Performance benchmarking — graph performance can be unintuitive; benchmark before committing.
• Maintainability — complex graphs become hard to understand. Document the graph structure.
• ffmpeg version pinning — graph behavior occasionally changes between versions.
• Graceful degradation — if filter_complex fails, what's the fallback? Plan error handling.

#Real-world production examples

Example 1: ABR ladder with watermark

ffmpeg -i master.mp4 -i logo.png -filter_complex \
  "[1:v]scale=120:60[logo]; \
   [0:v]split=3[v1][v2][v3]; \
   [v1]scale=1920:1080[full]; \
   [v2]scale=1280:720[hd]; \
   [v3]scale=854:480[sd]; \
   [full][logo]overlay=W-w-20:20[full_wm]; \
   [hd][logo]overlay=W-w-15:15[hd_wm]; \
   [sd][logo]overlay=W-w-10:10[sd_wm]" \
  -map "[full_wm]" -map 0:a -c:v libx264 -crf 22 -c:a aac full.mp4 \
  -map "[hd_wm]" -map 0:a -c:v libx264 -crf 22 -c:a aac hd.mp4 \
  -map "[sd_wm]" -map 0:a -c:v libx264 -crf 22 -c:a aac sd.mp4

Three ladder rungs with watermark applied at appropriate scale per rung.

Example 2: Per-shot dynamic processing

For content with scene changes that need per-segment processing:

ffmpeg -i input.mp4 -filter_complex \
  "[0:v]select='gt(scene,0.4)',showinfo[scene_info]; \
   [0:v]split=2[v1][v2]; \
   [v1]scale=1920:1080[full]; \
   [v2]scale=640:360[thumb]" \
  -map "[full]" -c:v libx264 -crf 22 full.mp4 \
  -map "[thumb]" -vf "fps=1" thumbnails/thumb_%03d.jpg

Generates main video plus periodic thumbnails plus scene-detect log.

Example 3: HDR-to-SDR with audio passthrough

ffmpeg -i hdr.mp4 -filter_complex \
  "[0:v]zscale=t=linear:npl=100,format=gbrpf32le, \
        zscale=p=bt709,tonemap=hable,zscale=t=bt709:m=bt709:r=tv,format=yuv420p[sdr_v]" \
  -map "[sdr_v]" -map 0:a \
  -c:v libx264 -crf 22 -c:a copy sdr.mp4

Tone-maps HDR to SDR; passes audio through unchanged.

#What MpegFlow does with filter_complex

MpegFlow's DAG runtime expresses filter graphs as parameters on FfmpegExecutor stages. Common operations are typed first-class — FilterParams::Deinterlace (yadif/bwdif), FilterParams::Thumbnail, FilterParams::Overlay for watermark/logo work — so most production graphs don't require hand-authored filter_complex strings; the runtime materializes the right invocation from the typed parameters. The partitioner persists each stage to job_stages with explicit dependency tracking; per-stage retry handles transient failures.

For customers needing graphs the typed parameters don't cover (specific composition patterns, time-windowed overlays, custom color processing pipelines), the workflow YAML supports passing through a custom filter_complex string at the FfmpegExecutor stage. That custom string is validated against the worker image's FFmpeg-build capability before promotion to production.

For ABR ladder generation specifically, the partitioner emits parallel sibling rendition stages rather than a single filter_complex split — the DAG runtime expresses fan-out at the orchestration layer, not inside one FFmpeg invocation, which keeps per-rendition retry, per-rendition cancellation, and rendition-level partial-success reporting clean.

The strict-broker security model handles filter_complex work like any pipeline payload — workers carry no ambient credentials; content access flows through short-lived presigned URLs scoped per stage; access is disposed on completion.

The general guidance: filter_complex is powerful but cryptic. Master a few common patterns (split, overlay, concat, hstack); build production graphs from those primitives. Don't try to construct from scratch under time pressure; reference proven patterns. Once you have a vocabulary of working filter graphs, building new ones becomes mechanical rather than artistic. Investment in the vocabulary pays off across the entire pipeline lifetime — the same fundamental graph patterns appear in ABR ladder generation, watermarking, color conversion, transcoding, and editorial workflows.