MiniMaxAI/MiniMax-H3

Overview

MiniMax H3 is an open-weight, general-purpose multimodal generation model. Rather than being confined to one specialized task — generate, edit, or reference — H3 reads a multimodal context that mixes text, images, video, and audio together, interprets the creative intent as a whole, and produces coherent audio-visual output end to end.

Architecturally it is a CFG-distilled joint video/audio diffusion transformer served through vLLM-Omni's OpenAI-compatible /v1/videos API. Every request returns a single MP4 containing H.264 video and native stereo audio — the audio is generated jointly with the video by the same DiT, not dubbed on afterwards.

Three capability areas the model targets:

• Commercial-grade generation — film and entertainment, advertising and branding, e-commerce, gaming; dynamic typography, VFX, product showcases, UI/UX motion design.
• Native multimodal understanding + generation — interprets characters, motion, sound, emotion, cinematography, and visual style across mixed references, then combines them.
• Precise multimodal editing and control — iterative refinement of characters, objects, scenes, sound, and rhythm with strong instruction following.

The checkpoint contains two DiT backbones that share the encoder and VAEs:

DiT	Tasks	Conditioning
FL2VA	t2va, fl2va	text only, or text + first/last frame
Ref2VA	ref2va	text + omni references (images, video clips, audio clips)

Since vLLM-Omni #5720, serving /path/to/MiniMax-H3 initializes both DiTs by default, while the Qwen3-VL encoder and video/audio VAEs stay shared. One server can therefore handle all three task types. Add --task-type fl2va or --task-type ref2va only when the hardware profile needs to load one DiT to save GPU or host memory.

Components (BF16): two 52-block joint video/audio DiTs (66.3 GB each), a shared Qwen3-VL layer-50 text/vision encoder (51.5 GB), a video VAE (~10 GB), and an audio VAE (~0.6 GB).

Model specifications

Output duration	4–15 seconds
Frame rate	24 FPS (fixed)
Audio	Native stereo, on every generation
Resolution — 1440p (2K)	Short edge 1440 px for 16:9 … 9:16; wider formats ≈3.7 MP (e.g. 2976×1248 at 21:9)
Resolution — 768p	Short edge 768 px for 16:9 … 9:16; wider formats ≈1 MP (e.g. 1536×672 at 21:9). Upscalable to 1440p.
Aspect ratio	First/Last Frame follows the uploaded image. Text-to-Video and Omni Reference select from 21:9, 16:9, 4:3, 1:1, 3:4, 9:16 — Omni Reference also offers Auto
Prompt length	Up to 7,000 characters

Input modes and limits

First/Last Frame — 0, 1, or 2 images. Edge lengths in [256, 5760], aspect ratio 5:2 to 2:5. With no image, H3 falls back to Text-to-Video.

Omni Reference — up to 9 images; up to 3 video clips (2–15 s each, 15 s total); up to 3 audio clips (2–15 s each, 15 s total). Edge lengths in [256, 5760], video aspect ratio 5:2 to 2:5. Audio must be paired with at least one image or video — it cannot be the sole reference. Mixed inputs cap at 12 files total; with none, H3 falls back to Text-to-Video.

	Formats	Per-file size
Video	H.264/AVC, H.265/HEVC (embedded audio AAC or MP3)	50 MB
Image	JPG, JPEG, PNG, WEBP, HEIC, HEIF	30 MB
Audio	WAV, MP3	15 MB

Limits are per file, not per combined upload. Keep the whole request body under 64 MB — prefer URL-based media over inlined uploads for anything large.

Prerequisites

H3 is served from a local directory, not the HF id. The root contains both DiT subdirectories plus the shared encoder and VAEs. Request access on the gated model card, then download the complete repository:

hf download MiniMaxAI/MiniMax-H3 --local-dir /path/to/MiniMax-H3

That yields /path/to/MiniMax-H3/FL2VA, /path/to/MiniMax-H3/Ref2VA, and the shared component directories. Serve /path/to/MiniMax-H3 itself and substitute your own --local-dir throughout the commands below.

ffmpeg and ffprobe must be on PATH — they are used for reference-video preparation and MP4 muxing.

Installation

Docker (recommended)

The image bundles H3 handlers, FlashAttention-4 kernels, and media dependencies. The published tag predates low-memory DLO and the modular pipeline merged in vLLM-Omni #5720, so clone current main before starting the server:

docker pull vllm/vllm-omni:minimax-h3
git clone --depth 1 https://github.com/vllm-project/vllm-omni.git /path/to/vllm-omni

The command builder's Docker mode also binds the source checkout read-only at the same absolute path and prepends it to any PYTHONPATH declared by the recipe, so current source takes precedence without generating duplicate environment flags:

-v /path/to/MiniMax-H3:/path/to/MiniMax-H3:ro
-v /path/to/vllm-omni:/path/to/vllm-omni:ro
-e PYTHONPATH=/path/to/vllm-omni

pip

H3 support ships in vLLM-Omni rather than the vllm wheel, so the pip path needs a source checkout:

uv venv
source .venv/bin/activate
uv pip install vllm==0.26.0
git clone --depth 1 https://github.com/vllm-project/vllm-omni.git /path/to/vllm-omni
cd /path/to/vllm-omni
uv pip install -e .

The RTX 5000/5090/4090 recipes below use CUDNN_ATTN and need no FlashAttention-4 install. For the four-GPU B300/GB200 FLASH_ATTN profile, add the CUDA-only CuTe-DSL kernels with uv pip install -e '.[fa4]'.

Launch — two 24/32 GB GPUs with distributed layerwise offload

vLLM-Omni includes a low-memory TP2 path for H3. Install the current vLLM-Omni source checkout as shown above.

The two-GPU profile combines TP2 with distributed layerwise offload (DLO). Each rank keeps its rank-local TP shard in pinned host memory; --dlo-no-use-allgather streams those shards directly instead of reconstructing full DiT blocks on every rank. A configurable prefix of DiT blocks stays resident for the complete denoise stage, then releases its HBM before VAE decode.

Select the profile first. RTX 5090 uses 20 resident DiT layers at the released 1344×768 shape; RTX 4090 keeps more HBM headroom with 12 resident layers and a 1024×576 starting shape:

Profile	GPUs	Starting shape	Resident DiT blocks	Attention	Execution	Status
RTX 5090	2 x 32 GB	1344x768	20	cuDNN attention	eager	Target-hardware validated
RTX 4090	2 x 24 GB	1024x576	12	cuDNN attention	eager	Capacity-proxy starting point

This topology uses all available parallel capacity: TP2 shards both the DiT and text encoder, --dlo-no-use-allgather streams each rank's local TP shard without reconstructing full blocks, and VAE patch parallelism splits tiled decode across both GPUs. cuDNN attention is selected explicitly for the RTX consumer path; the server stays eager to avoid an unqualified compile path.

# 2× RTX 5090 (32 GB)
export DLO_RESIDENT_LAYERS=20 WIDTH=1344 HEIGHT=768

# Or, 2× RTX 4090 (24 GB)
# export DLO_RESIDENT_LAYERS=12 WIDTH=1024 HEIGHT=576

Load only the FL2VA DiT on these memory-constrained profiles:

CUDA_VISIBLE_DEVICES=0,1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --task-type fl2va \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 2 \
  --tensor-parallel-size 2 \
  --usp 1 \
  --ring 1 \
  --text-encoder-tp-size 2 \
  --vae-patch-parallel-size 2 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --enable-distributed-layerwise-offload \
  --dlo-no-use-allgather \
  --dlo-resident-layers "${DLO_RESIDENT_LAYERS}" \
  --enforce-eager \
  --diffusion-attention-backend CUDNN_ATTN

The resident count changes placement and transfer frequency only; it does not quantize or change the BF16/FP32 denoise math. Use a 384 GiB-class host for the pinned model shards and offload buffers.

RTX 5090 target-hardware validation

At vLLM-Omni commit ae6577ea, one full 50-step T2VA request completed on 2 x RTX 5090 without OOM:

Shape	Frames	Client E2E	Sampled peak/GPU	Output validation
1344x768	124 at 24 FPS	8 min 38 s	approximately 22.6 GiB	H.264 video + 32 kHz stereo AAC; full ffmpeg decode passed

This is a single end-to-end validation run, not a warmed multi-run latency benchmark. The sampled nvidia-smi peak is also not a CUDA allocator high-water mark. The environment used vLLM 0.26.0, vLLM-Omni 0.26.1.dev14+gae6577ea, and PyTorch 2.11.0+cu130. The run record captures the environment, output contract, elapsed time, and sampled peak.

The RTX 4090 profile remains a capacity-proxy starting point rather than a target-hardware latency claim. Measure its peak HBM and E2E latency on the exact request shape before presenting it as performance-qualified.

Keep 20 resident layers as the released RTX 5090 default. On a matched two-rank B300 capacity proxy, increasing the count to 24 reduced the median 5-step client E2E from 14.91 s to 14.01 s while raising the sampled peak from 26,310 MiB to 28,774 MiB per GPU. The 20- and 24-layer outputs were byte-identical. A 50-step 24-layer request completed in 93.36 s client E2E (91.82 s engine) at a 28,760 MiB peak and produced a fully decodable H.264/AAC MP4. These are B300 proxy measurements, not RTX latency validation; use DLO_RESIDENT_LAYERS=24 only after confirming at least 3 GiB of free HBM on the exact RTX request shape.

Wait for Application startup complete, then verify health and run one full 50-step T2VA request:

curl --fail http://127.0.0.1:8000/health

curl --fail-with-body -sS \
  -X POST http://127.0.0.1:8000/v1/videos/sync \
  -F 'prompt=At night, three cats march into a bedroom playing tiny brass instruments, then abruptly file out, with synchronized room ambience.' \
  -F "width=${WIDTH}" \
  -F "height=${HEIGHT}" \
  -F 'fps=24' \
  -F 'num_inference_steps=50' \
  -F 'flow_shift=12' \
  -F 'seed=1101' \
  -F 'extra_params={"task":"t2va","duration":5.0,"audio_flow_shift":3.0}' \
  -o minimax-h3-t2va.mp4

ffprobe -v error -show_entries \
  stream=index,codec_name,width,height,r_frame_rate,sample_rate,channels \
  -of json minimax-h3-t2va.mp4
ffmpeg -v error -i minimax-h3-t2va.mp4 \
  -map 0:v:0 -map 0:a:0 -f null -

A successful output has H.264 video at 24 FPS and stereo AAC audio at 32 kHz, and the final ffmpeg command exits with status 0. This capacity profile intentionally loads one DiT. Restart the same root-model command with --task-type ref2va for Ref2VA; remove --task-type only after confirming that the combined modular service fits the target.

For repeatable T2VA, FL2VA, Ref2VA, peak-memory, and profiler collection, use the MiniMax-H3 matrix runner.

Launch — four GPUs

B300 / GB200 (NVLink)

The datacenter default uses no offload, Ulysses SP4, native tiled VAE PP4, regional torch.compile, FA4, and Ring/TP1. The root-model launch below uses the modular service: both DiTs are initialized once while the encoder and VAEs are shared.

CUDA_VISIBLE_DEVICES=0,1,2,3 \
FLASHINFER_DISABLE_VERSION_CHECK=1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 4 \
  --usp 4 \
  --ring 1 \
  --vae-patch-parallel-size 4 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --diffusion-attention-backend FLASH_ATTN

RTX PRO 5000 Blackwell (SM120, PCIe, no NVLink)

The validated resident profile is TP2 × Ulysses2, text-encoder TP4, VAE PP4 tile, cuDNN attention, BF16, and no offload. On the tested dual-socket eight-GPU host, CUDA_VISIBLE_DEVICES=0,2,1,3 maps Ulysses rank pairs onto the local PXB pairs (0,1) and (2,3); adapt the IDs after checking nvidia-smi topo -m on another host. Bind CPU and memory to the NUMA node local to the selected quartet.

CUDA_VISIBLE_DEVICES=0,2,1,3 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
numactl --cpunodebind=0 --membind=0 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --task-type fl2va \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 4 \
  --tensor-parallel-size 2 \
  --usp 2 \
  --ring 1 \
  --text-encoder-tp-size 4 \
  --vae-patch-parallel-size 4 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --diffusion-attention-backend CUDNN_ATTN

This 72 GB/GPU profile intentionally selects one DiT. Use --task-type ref2va in a separate process for Ref2VA; loading both DiTs together is not the qualified 5000 profile. A five-step topology screen (after warmup) produced decodable H.264 + stereo AAC outputs:

Workload	Client E2E	Encode	Denoise	VAE decode	Peak/GPU
T2VA, 124 frames, 1344×768	32.993 s	0.034 s	27.915 s	4.307 s	67.6 GiB external maximum
FL2VA first frame, same output shape	30.130 s	0.276 s	24.396 s	4.339 s	67.6 GiB external maximum

These are short topology-screening numbers, not a 50-step throughput claim. The external nvidia-smi sampler reported physical-GPU peaks of 67,681 / 69,143 / 69,207 / 69,207 MiB. An Nsight Systems run of the same topology showed:

Workload	AllGather	Send/Recv	NCCL other	NCCL total	Dense FMHA	Load balance (max-min/mean)
T2VA	0.93%	8.53%	14.63%	24.09%	31.15%	1.11%
FL2VA	0.89%	7.26%	15.76%	23.91%	32.53%	0.13%

NCCL other is the analyzer bucket for NCCL kernels that are neither recognized AllGather nor point-to-point Send/Recv; it includes collectives such as AllReduce, ReduceScatter, and Broadcast when present. Keep it visible when comparing against FSDP.

Experimental SM120 FP8 CuTe DSL prims

The default above remains CUDNN_ATTN. Draft vLLM-Omni #5860 exposes the FlashInfer SM120 CuTe DSL kernel through TRTLLM_ATTN; it is an opt-in FP8 attention experiment, not a released recipe default. Build the exact tested FlashInfer revision and Draft PR:

git clone --recursive https://github.com/Tom-Zheng/flashinfer.git /path/to/flashinfer-sm120
cd /path/to/flashinfer-sm120
git checkout 4a2345906256da0849d7e1e4681db514ab9b800e
python -m pip install -v '.[cu13]'
python -m pytest -q tests/attention/test_sm120_prims_prefill_backend.py

cd /path/to/vllm-omni
git fetch https://github.com/vllm-project/vllm-omni.git pull/5860/head
git checkout --detach FETCH_HEAD
uv pip install -e .

Replace the cuDNN backend flag with:

--diffusion-attention-config '{
  "default": {
    "backend": "TRTLLM_ATTN",
    "quant": {
      "dtype_qk": "fp8_e4m3",
      "dtype_vo": "fp8_e4m3",
      "flashinfer_backend": "cute-dsl-prims"
    }
  },
  "per_role": {
    "minimax_h3.token_refiner": {"backend": "TRTLLM_ATTN"}
  }
}'

Omitted Q/K/V scales are calibrated once on the first call, so warm up the same engine before measuring. This path requires SM120, fails explicitly on B300/SM103, and still needs a matched 50-step latency and video/audio fidelity comparison before promotion.

AMD ROCm — four MI300X GPUs

Verified on 4× AMD Instinct MI300X (gfx942) with the ROCm H3 image, AITER packed variable-length attention, Ulysses 4, text-encoder TP4, and native tiled VAE patch parallelism 4:

docker pull vllm/vllm-omni-rocm:minimax-h3

HIP_VISIBLE_DEVICES=0,1,2,3 \
VLLM_ROCM_USE_AITER=1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 4 \
  --usp 4 \
  --ring 1 \
  --text-encoder-tp-size 4 \
  --vae-patch-parallel-size 4 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --diffusion-attention-backend FLASH_ATTN

On ROCm, FLASH_ATTN resolves to AITER on supported Instinct architectures. Do not install the CUDA-only [fa4] extra or set FLASHINFER_DISABLE_VERSION_CHECK.

The refreshed vllm/vllm-omni-rocm:minimax-h3 tag includes the Ref2VA soundfile fallback, TorchCodec, and FFmpeg, so T2VA, FL2VA, and the complete Ref2VA path use the same official image.

AMD ROCm — four MI355X GPUs

Verified on 4× AMD Instinct MI355X. The official ROCm image includes the MiniMax H3 Ref2VA fixes, TorchCodec, and FFmpeg. We tested both Ulysses 4 with Ring 1 and Ulysses 1 with Ring 4. The command below shows the Ulysses 4 / Ring 1 configuration with tiled VAE patch parallelism degree 4, AITER, and CPU offload:

docker pull vllm/vllm-omni-rocm:minimax-h3

HIP_VISIBLE_DEVICES=0,1,2,3 \
VLLM_ROCM_USE_AITER=1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 4 \
  --usp 4 \
  --ring 1 \
  --vae-patch-parallel-size 4 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --enable-cpu-offload \
  --text-encoder-tp-size 4

This root-model command uses the modular service and accepts T2VA, FL2VA, and Ref2VA requests without restarting.

Three constraints on this configuration:

• Do not add --enforce-eager. Regional compile is the default (--diffusion-compile-granularity regional) and is worth ~9.5%. The first request includes compilation — warm the server once before measuring steady state.
• --cfg-parallel-size must stay 1. H3 is CFG-distilled and has no negative branch; anything greater is rejected with an explicit error rather than silently duplicating a branch that does not exist.
• The VAE supports its native tile mode only — not spatial_shard_width or spatial_shard_height (those belong to the distributed Wan autoencoder). Patch parallel size must be 1 or the full DiT group size.

Launch — single GPU (accuracy and memory first)

Model-level CPU offload keeps the Qwen3-VL encoder and the DiT from being resident simultaneously. This is the accuracy-qualified reference path; it trades PCIe/NVLink transfer latency for a much smaller GPU footprint, and needs enough system RAM for the offloaded components.

CUDA_VISIBLE_DEVICES=0 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve /path/to/MiniMax-H3 \
  --omni \
  --task-type fl2va \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --num-gpus 1 \
  --enable-cpu-offload \
  --diffusion-attention-backend FLASH_ATTN

Text-encoder tensor parallelism

By default the whole Qwen3-VL encoder is resident on the DiT main rank, making it the peak-memory hotspot on multi-GPU no-offload runs. --text-encoder-tp-size N shards it across the first N DiT ranks using vLLM-style tensor-parallel layers on a dedicated encoder process group:

vllm serve /path/to/MiniMax-H3 \
  --omni --trust-remote-code \
  --task-type fl2va \
  --num-gpus 4 --usp 4 --ring 1 \
  --text-encoder-tp-size 4 \
  --vae-patch-parallel-size 4 --vae-parallel-mode tile --vae-use-tiling \
  --diffusion-attention-backend FLASH_ATTN

• N must divide the Qwen3-VL head counts (64 attention heads / 8 KV heads) → 1, 2, 4 on a 4-GPU server; 1, 2, 4, 8 on 8 GPUs.
• On 4 GPUs, N=4 drops the per-rank engine peak from 133 GB to 103 GB (~35.7 GB) at no measurable throughput cost, while each non-main rank gains ~51.5/N GB.
• Numerically: N=1 reproduces the HF reference path bit-exactly (max_abs = 0); N=4 introduces bounded BF16 rounding only (row-parallel all-reduce in FP32), giving 31.11 dB PSNR / 0.9566 SSIM end-to-end against N=1 with identical structure.
• Combining N>1 with regional compile used to produce a ValueError: v must be finite on the second request. That is fixed — the token refiner's attention runs on replicated rows before sp_prepare and was wrongly getting a Ulysses SP all-to-all.

Key request parameters

Parameter	Recommended	Notes
task	t2va, fl2va, ref2va	Passed inside extra_params; must be compatible with the loaded --task-type when one is set
duration	4–15 s	Decimal seconds in extra_params; snapped to H3's legal 17n+5 frame count (15 s → 362 frames / 15.083 s)
fps	24	Output FPS is fixed
num_inference_steps	50	Matches the reference accuracy workloads
flow_shift	12	Video sigma shift
audio_flow_shift	3	Audio sigma shift, in extra_params
width, height	see below	Server-side validation: multiples of 32, aspect ratio 1:4 to 4:1
seed	fixed value	Output is deterministic per request at a fixed seed

On width/height, the serving-side check is broader than the documented product modes — pick a short edge of 768 or 1440 in one of the supported ratios (21:9, 16:9, 4:3, 1:1, 3:4, 9:16) to stay on shapes the model was trained and validated for. FL2VA can omit both and inherit the first frame's aspect ratio at a 768 px short edge. Prompts cap at 7,000 characters.

Use POST /v1/videos/sync when you want the MP4 in the response body; POST /v1/videos is the async job-polling variant.

Validated performance (4×B300)

The first table used task-specific single-DiT processes before the modular combined-service merge; it remains the validated per-task operating point.

Workload	Config	Result
FL2VA, 209 frames, 1248×768 (8.7 s)	no offload, U4, VPP4 tile, regional compile	86.96 s client E2E (±0.23 s)
Two-video Ref2VA, 362 frames, 1344×768 (15 s)	same	784.4 s accounted model-stage mean

Stage split for the FL2VA case: text encoder 0.21 s, visual encoder 0.20 s, DiT 79.1 s (88% of the request), video+audio VAE decode 2.40 s. VAE patch parallelism is the cheapest win available — it cuts decode 3.4–3.5× (8.24 s → 2.40 s).

B300 official Starship attention matrix

A separate strict run used the official 1344×768, 243-frame (10 s), 50-step Starship prompt on physical GPUs 0, 1, 4, and 5. Each mode used one regional-compile warmup and five serial measured requests; the table reports median diffuse time, not E2E:

Backend	Attention optimization	Median diffuse	Speedup vs dense	CV
TRTLLM_ATTN	dense BF16	109.347 s	1.000×	0.12%
TRTLLM_ATTN	FP8 SAGE	90.014 s	1.215×	0.17%
TRTLLM_ATTN	Skip-Softmax 0.05, gate 0.97 (35/49 steps)	101.734 s	1.075×	0.10%
TRTLLM_ATTN	FP8 SAGE + same Skip-Softmax	86.983 s	1.257×	0.13%

All modes passed the strict timing gate (CV below 2%, span/median below 5%, no thermal slowdown). SAGE and Skip-Softmax change attention math and are not the accuracy-qualified default. See the videos, JSON records, timing results, and thermal audit.

Validated performance (4×MI300X)

Measured with vllm/vllm-omni-rocm:minimax-h3 (sha256:29d1946af9c69e3e0a7128c247bdc8c82437ead43fbc366fed443533cf6ce9e8), BF16, AITER, U4, text-encoder TP4, VPP4 tile, regional compile, one warmup, and the synchronous Video API:

Workload	Model stages	Client E2E
T2VA, 209 frames, 1344×768 (8.7 s)	encode 0.09 s, denoise 244.04 s, decode 4.15 s	267.42 s
FL2VA, 209 frames, 1344×768 (8.7 s)	encode 13.98 s, denoise 257.58 s, decode 4.11 s	287.07 s

Both outputs were validated as 209-frame H.264 at 24 FPS with 32 kHz stereo AAC. A v0.26.0 source build additionally completed image+audio and video-reference Ref2VA, plus a 2048×1088 T2VA run. These measurements describe tested shapes, not a general latency guarantee.

A with_stack profile of the two-video Ref2VA case shows the pipeline is GPU-attention-bound, not host-starved: FlashAttention-4 is ~76% of diffuse device time, Ulysses send/recv is secondary, GPU utilization is 92.8–94.3% and the CPU-idle union inside the transformer forwards is 3.15%. Reducing DiT time further needs a change in attention complexity (reference-latent pooling/pruning, block-sparse attention), which is not lossless.

Against the checkpoint's own reference implementation, the accuracy-qualified path scores SSIM 0.9873–0.9896, PSNR 39–42 dB, pixel cosine 0.9996+, and audio log-mel cosine 0.977–0.996 on T2VA/FL2VA. The two-video Ref2VA composite scores lower on raw pixels (SSIM 0.628) largely because of H.264 bitrate differences, while CLIP cosine is 0.9816 and audio log-mel cosine 0.9869 — content and motion stay aligned.

Cache-DiT (experimental)

H3's DiT supports Cache-DiT. To enable it, append the following flag to the selected vllm serve command above:

--cache-backend cache_dit

This is an opt-in, lossy optimization and is not part of the accuracy-qualified deployment profiles. A 50-step T2VA probe cut denoise time 30.2% (121.01 s → 84.47 s) but dropped SSIM to 0.831 against the uncached output. H3's joint video/audio trajectory is sensitive to cached denoise steps, so compare both video and audio against an uncached run before using it in production. The probe above covers T2VA; validate FL2VA and Ref2VA separately for your workload.

TeaCache is not currently supported for MiniMax H3. Do not use --cache-backend tea_cache, and do not combine TeaCache with Cache-DiT.

Offload and accuracy

On a single GPU, model-level CPU offload is the accuracy-qualified path. On four GPUs, no-offload is both faster and accuracy-clean once the encoder stays resident: an earlier no-offload build performed a ~63 GiB DiT GPU→CPU→GPU round trip inside encode_prompt, which was pure overhead (encoder 30.38 s → 0.207 s once removed, ~146×).

Known limitations

• The modular root service loads both DiTs by default. Capacity profiles use --task-type fl2va or --task-type ref2va and therefore expose only that task family.
• H3 executes one generation request per diffusion batch today.
• The first request is a compile warmup only on the resident regional-compile profiles. The two-GPU DLO recipes use --enforce-eager and do not compile the DiT.
• The serving path accepts fewer references than the model supports. H3 documents up to 9 images, 3 video clips, and 3 audio clips (12 files) per Omni Reference request; the current vLLM-Omni path takes exactly one image plus one audio reference, or one or more videos with no separate audio_reference (it uses the source soundtracks).
• The 768 px short-edge mode is available for T2VA and FL2VA; 1344×768 is the documented 16:9 request shape. The 24 GB DLO recipe uses 1024×576 as its lower-memory starting point.
• --cfg-parallel-size > 1 is rejected by design (CFG-distilled, no negative branch).
• The VAE supports the native tile parallel mode only.
• A U2 × Ring2 hybrid currently fails with an attention-mask length mismatch; use pure Ulysses.
• Full-transformer FP8 is not supported yet. B300 SAGE and the Draft SM120 CuTe DSL path quantize attention activations only and require separate fidelity qualification.
• Pure Ulysses still replicates the 66.3 GB DiT on every rank, so 64 GB GPUs cannot use --usp N --tp 1 as a resident capacity path. Use DiT TP plus DLO (the two-GPU recipes above), or model-level CPU offload; text-encoder TP alone is not sufficient.
• DLO is a capacity path, not a free speedup: it needs substantial host RAM and streams weights over PCIe on every denoise step. Increase --dlo-resident-layers only after checking peak HBM on the exact request shape.

References

• Model card
• Modular FL2VA/Ref2VA pipeline (vLLM-Omni #5720)
• Experimental SM120 CuTe DSL attention (Draft #5860)
• B300 official Starship matrix
• RTX PRO 5000 matrix and profiler runner
• vLLM-Omni Video API
• vLLM-Omni diffusion parallelism
• vLLM-Omni diffusion attention backends
• vLLM-Omni diffusion cache acceleration
• vLLM-Omni supported models
• vLLM-Omni GPU installation
• ROCm H3 tracking issue