本文贡献:QASK-Edit(Query-Adaptive Sparsity over Key Sources for Video Editing)是无需再训练的稀疏注意力路由:先汇总块与子块,再对每个查询 tile(计算分块)、每类 key 来源分别计算注意力质量和近似风险,在各自风险预算内选阈值,最后把来源合入同一次 softmax。
Figure 2 : Overview of QASK-Edit. The process begins with the generation of summaries for blocks, sub-blocks, and second-level sub-blocks. For each query tile, we independently select a routing threshold for each key source within the risk budget, based on the residual risk introduced by approximate calculations. This routing threshold then determines whether to perform exact or approximate calculations. Finally, we aggregate the contributions of all key sources into a single softmax and output the final result.
Figure 1 : Qualitative video editing results of QASK-Edit on MiniMax-H3. Three sampled frames are shown for each source–edited video pair, covering diverse editing tasks such as clothing modification, object replacement, and scene recoloring. The results demonstrate high-quality edits with preserved temporal consistency.
Figure 2 : Overview of the two-stage training pipeline of LiveView. Stage 1: We train a bidirectional camera-controlled V2V teacher, where Cross-Frame Injection interleaves synchronized source–target latents and camera ray embeddings. Stage 2: We distill the teacher into a few-step causal DiT via distribution matching distillation (DMD), enhanced with Cycle Streaming Tuning and Shared Relative RoPE (SR-RoPE) for stable long-horizon streaming generation.
方法直觉:最初几帧是尚未漂移的外观锚点。让后续区域专门对齐对应的初始区域,再把缓存中漂移的 value 拉回初始外观,可减少误差累积。
本文贡献:Reference Forcing 先用现成分割器给教师建立主体与背景区域的锚定注意力,再将行为蒸馏进学生的独立注意力支路;推理时无需外部分割器。第二步 value correction 按 key 匹配,把已漂移片段的 KV 缓存 value 向初始片段校正。KV 是注意力缓存的键和值。
Figure 2: Overview of Reference Forcing. (a) Each chunk X n X_{n} is generated from the first chunk X 0 X_{0} , the recent window, and noisy tokens, with the anchor path of (b) and the value correction of (c). (b) In every layer, the student adds an anchor path (LoRA on the query and keys, and the anchor gate w w ) to the frozen native attention and learns to match the prediction of a teacher whose anchor path is masked by the subject mask and the background flow. (c) Value correction moves the values of the drifted chunk X n − 2 X_{n-2} toward their key-matched values in X 0 X_{0} , with the anchor gate w w learned in (b) as strength.
Figure 3: Qualitative comparison. All methods share the prompt and first chunk; boxes mark subject changes, magnified in the zoom column. Left: the dark visor of the astronaut turns transparent in baselines but stays dark in Reference Forcing. Right: in baselines, the face of the woman becomes lighter and the strap on her left shoulder wider and brighter, whereas Reference Forcing keeps both.
本文贡献:RAGE(Residual-Adaptive Gating for Flow-Based Image Editing)在每次采样后做一次轻量门控插值,把小残差潜变量拉回源图,同时放行大的语义编辑。它叠加在 FlowEdit 类编辑器上,不需估计蒙版、重训模型或改架构。
Figure 1: Without RAGE (top), the base editing velocity V θ V_{\theta} causes unintended drift in non-editing regions that accumulates over timesteps. With RAGE (bottom), the source-aware correction λ ∇ R \lambda\nabla R selectively suppresses small deviations while leaving large, meaningful edits unimpeded.
Figure 2: Qualitative comparison on PIE-Bench with FLUX.1-dev. From left to right: Source, DNAEdit-FLUX, FireFlow-FLUX, FlowEdit-FLUX, and Ours (FLUX). Source and target prompts appear below each example, with the changed concepts highlighted. Red boxes mark unintended changes outside the edited region. The selected rows cover shape modification, object replacement, color editing, object addition, and expression editing.
Figure 3: Overview of Découpage . (a) Dual-track temporal cross-attention guides joint audio-video generation along independent visual and acoustic schedules, allowing dialogue to continue across shot boundaries. (b) The directorial lip-sync mechanism blends speech and no-speech A2V residuals based on speaker visibility to suppress unintended listener articulation.
Figure 4: Qualitative comparison on a five-shot screenplay. Columns present shot prompts and the target dialogue schedule; rows show keyframes and synthesized audio waveforms across models. Blue/orange highlights mark each character’s speech windows, and gray denotes silence. Découpage adheres to the multi-shot structure across transitions, respects the Shot-4 listener, and correctly synthesizes the Shot-5 slience and lighter flick, achieving superior audio-visual alignment.
Figure 3: From authored assets to play. Left: authored once, a first frame and a character sheet, and a skill tree that fuses an attribute of the sheet (fire) with each mechanic. Middle: at run time the GameWM-engine renders low-poly proxy frames and resolves the event, here a ranged attack that becomes Fireball. Right: the four inputs fill the slots of the GameWM-VM’s reference interface (H3’s Ref2VA, Section 6 ); the output column shows frames of a BD-Sheet sample generated from them.
Figure 8: One mechanic, many realizations: ranged and melee. The left four columns are the shooting clip and the right four a melee clip; each clip keeps one capsule proxy and one caption template (the box above its columns). Each row is one character: its sheet and name (far left), a first frame of that character in each scene, and its skill for each mechanic (the tag on the first frame of each clip). BD-Sheet, 30 steps; red ticks mark the shot, the strike, and the fall as read off the proxies. The DMD-Sheet version of the ranged clip is in Figure 13 .
本文贡献:ReMoVe 从基础视频及首帧头像恢复动作并修正,渲染三维人体网格、迁移脸部外观,并由相机轨迹推导深度;将网格与深度视频编码后连同文字输入 DiT 重生成。DiT 是以 Transformer 为骨干的扩散生成器。
Figure 2: Overview of ReMoVe. On the left is the construction of mesh and depth conditions during training (top) and inference (bottom). On the right is the generation pipeline, where the mesh and depth videos are encoded by the VAE, concatenated with the noisy video latent, and fed into the DiT together with the text prompt to produce the final high-quality video.
实验效果:在 Wan-TI2V-5B 人体视频设置中,网格加深度版的人体解剖指标 HA 为 92.66,对基础 Wan 的 90.88;视频分布距离 FVD-16 为 69.87,对基础 Wan 的 91.41。HA 越高越好,FVD-16 越低表示生成分布更接近参考。
Figure 4: Qualitative comparison of human video generation. ReMoVe preserves the original motion prior of Wan while substantially improving human structural stability, appearance consistency, and temporal coherence, outperforming prior methods on challenging human motions.
Figure 2 : The top shows the overall inference pipeline, integrating denoising acceleration and KV cache optimization. The bottom illustrates three components. (a) CDR estimates clean latent updates from noise variations and shallow feature changes. (b) ORHR allocates historical KV reading budgets based on attention output perturbations. (c) CCSC reduces cache construction costs through content and coverage guided spatial token selection.
Figure 7 : Mechanism analysis of ORHR. (a) Unequal historical KV dependence across Transformer layers and heads. (b) Attention output errors under uniform reading and ORHR with the same budget of 720 frame accesses. (c) Output NMSE across subsequent video chunks using the early calibrated allocation, demonstrating consistently lower errors without recalibration.
Figure 3 : Overview of DuoFlow. The selected motion instruction, transcript, and voice description form a shared semantic condition for two coupled latent streams. A frozen phoneme encoder supplies a separate audio condition. Its normalized phoneme-progress anchor preserves token order, and phoneme-CTC heads supervise the audio states at multiple depths during training only. Each dual-stream block combines self-attention, condition cross-attention, phoneme cross-attention, and bidirectional speech–motion exchange. Frozen decoders reconstruct the denoised latents as a waveform and a motion sequence.
Figure 4 : Qualitative comparison under hierarchical motion control. The upper block shows generated waveforms, time-aligned transcript words, and sampled motion frames under the same coarse L1 motion instruction. The lower block shows the results after changing the motion description to a fine-grained, temporally ordered L2 instruction. Red words in the time-aligned transcripts mark mismatches with the input text.
评论 (0)