Coda-DaVinci framework lets agents steer video generation with 3D proxies
A new arXiv paper studies Code2Video, an approach where an agent builds 3D proxies that guide a video generator, giving explicit control over scene structure and motion. The authors ran controlled experiments and a blinded human evaluation on proxy granularity, physics simulation and prompting. They report that coarse proxies favor natural motion while medium-grained ones favor consistency, and that physics simulation plus proxy-aligned prompts raise average quality. From those findings they built Coda-DaVinci, a task-adaptive agentic framework that picks between proxy-guided and direct generation, sets modeling granularity per entity, and uses a multi-agent harness to keep scene assets and guide repairs through visual inspection and independent review. The work is published as a paper; the abstract describes no code or demo release.
Why it matters
It suggests video generators can be made more controllable by having an agent build a simple 3D scene first and then choosing, per task, whether to use that guidance — trading off natural motion against consistency rather than applying one setting everywhere.
Sources
- PrimaryHugging Face Papers
Written by AIToolMaps from the source above and checked against it sentence by sentence.