Black Forest Labs’ FLUX 3 adds video, audio and motion to diffusion
Synthesized by Helena Ross, Strategic Think-Tank Desk
- FLUX 3 is trained on video, audio and still images to capture multi-modal cues.
- Its video upscaler shares the FLUX 3 backbone, improving temporal consistency.
- A short fine-tuning run on a small labeled set led to a larger investment in data and model scaling.
Black Forest Labs explains how training on synchronized video, audio and still images expands FLUX 3’s visual capabilities.
«Building Visual Intelligence — Andreas Blattmann & Dustin Podell, Black Forest Labs»
Summary and commentary by Aura-V. Facts, figures and opinions belong to the original.
Multi-modal training as a new frontier
Andreas Blattmann and Dustin Podell argue that a model that sees a still image, a moving camera and a soundtrack learns a form of “visual intelligence” that single-frame diffusion cannot capture. Their FLUX 3 release, detailed in a Latent Space essay, trains on three synchronized signals, letting the model infer object geometry from motion and timing cues from sound.
Temporal consistency through shared backbones
FLUX 3’s video upscaler reuses the same diffusion backbone, preserving temporal coherence even when users request higher resolution. This engineering choice demonstrates that consistency is a design goal, not an after-thought.
Where compute meets product decisions
Dustin’s early Kontext experiment illustrates the lab’s allocation of resources. A training run of a few hours on a small set of photos labeled with relationships got the model to put a logo on a shirt. It was rough, but it was enough to take to the next offsite, after which the team spent months on the data and the model.
Unanticipated user value
A user story near the end describes people feeding FLUX old, degraded family photos, restoring them, then showing the pictures to their families. Dustin says he hadn’t predicted that use. The anecdote highlights how preserving identity across edits and frames becomes a litmus test for the model’s fidelity.
Our read: The conversation positions FLUX 3 as an effort to extend diffusion beyond static images by incorporating video and audio, betting that the added complexity will unlock use cases that static images cannot satisfy, while acknowledging that commercial success depends on unpredictable user adoption.
The interview discusses adding video and audio to diffusion models, noting both technical promise and the practical unpredictability of user-driven use cases. It does not claim a clear shift away from static diffusion.
Interested in implementing this technology inside your company?
Our studio designs custom agent meshes, data pipelines, and outbound GTM systems for forward-looking enterprises.