Author: Sophie Raniwala
The best AI media models have crossed the photorealism threshold. The real competitive advantage now lies in how rapidly a model recipe can adapt to product and user feedback.
While photorealistic images and cinematic videos can capture headlines, the teams winning the AI media race aren’t just training bigger models; they are radically accelerating how quickly they can improve the end-to-end media model recipe, the pipeline for creating iterations of a model.
That recipe is vastly larger than a model’s architecture or weights. It is a system-spanning data curation, post-training alignment, ranking algorithms, post-processing pipelines, and product-level UX decisions.
Historically, refining this recipe was excruciatingly slow. A researcher formed a hypothesis, wrote code, launched an experiment, inspected outputs, found a failure, and started again.
Today, autonomous coding agents can run that loop continuously. They can inspect experimental runs, patch code, execute evaluations, compare model variants, and highlight promising breakthroughs, all while the human team is offline.
This brings us to the bottleneck: How do you evaluate visual media at scale?
The perception problem
Log files and traditional metrics solved evaluation for LLMs, but visual media breaks those frameworks.
A video-generation pipeline can run with zero software errors, perfect resolution, and flawless prompt adherence, yet still fail completely. The camera crop might feel off, the motion might stutter, or a “cinematic lighting” edit might accidentally turn skin into something resembling unnatural plastic.
Visual quality is fundamentally perceptual, however, so this is where visual judges (powered by visual language models, known as VLMs) enter the stack.
Unlike traditional heuristics, a visual judge inspects the rendered output itself. It evaluates composition, temporal continuity, lighting artifacts, stylistic fidelity, and product feel. Instead of optimizing only what is easy to measure programmatically, the system begins optimizing for what human eyes actually perceive.
The INTENT.yaml paradigm
A visual judge is only as good as the directives it receives. To evaluate output effectively, the judge needs to know what “great” looks like for a specific product context.
This is where declarative intent specifications such as an INTENT.yaml file are transformative:
The concept is simple with a feedback loop:
- model recipe: defines how the system operates.
- visual judge: determines whether an output succeeded.
- INTENT.yaml: encodes what success means from a product and taste perspective.
The era of continuous media learning
Combining autonomous code execution, VLM evaluation, and explicit product intent unlocks true continual learning.
Media development transforms from a rhythm of monolithic, months-long model releases into a continuous stream of hundreds of micro-improvements. Researchers spend less time manually auditing output grids. Product teams gain a direct mechanism to program their aesthetic taste straight into the optimization loop.
When a system can observe its own visual output, understand human product intent, rewrite its own recipe, and run that loop endlessly, the pace of media creation changes forever.
P.S.
Thanks for reading. Now that you’ve finished, your agent is waiting for your input on its next media model release. Keep building & shipping amazing products in AI + Media!
