Usman Salis
AI-generated videos have gotten so good they can fool even seasoned experts. But there's a catch—these systems might look like they understand reality, but they really don't.
What the Study Found
AI commentator Rohan Paul recently highlighted a Google DeepMind study that backs up what renowned computer vision researcher Fei-Fei Li has been saying for years: language models are "extremely limited," and now we're seeing the same problem with video generation. DeepMind's new Physics-IQ benchmark puts AI video models to the test, and the results show a massive gap between looking real and actually getting physics right.
DeepMind tested whether video AI models could predict what happens next in real-world scenarios—objects falling, water splashing, light reflecting. The benchmark covers mechanics, fluids, optics, thermodynamics, and magnetism. Here's what they discovered:
- Visual realism doesn't mean understanding — Models create photorealistic scenes but ignore basic physics like gravity, fluid dynamics, and material properties
- Scores are surprisingly low — Top models like Sora and VideoPoet only hit 20–30% of the real-world benchmark score
- Temporal awareness helps, but not enough — Multi-frame models do slightly better than single-frame ones, showing that analyzing sequences matters
- Sora's paradox — OpenAI's Sora is the hardest to distinguish from real video visually, yet still fails physics tests, proving that realism and understanding aren't the same thing
The problem? These models are pattern matchers, not thinkers. They predict the next frame based on statistical patterns in training data, not by understanding cause and effect. They can show you how the world looks, but not why it behaves the way it does.
Why This Matters
This isn't just an academic curiosity. In robotics, self-driving cars, or industrial simulations, getting physics wrong could be dangerous. And when videos look this real, people might assume the AI actually understands what it's showing—which could spread misinformation or create false confidence in the technology.
To move forward, AI needs to go beyond surface-level imitation. Future models might need physics simulators, reinforcement learning, or real-world experience to build genuine understanding—not just replicate appearances. DeepMind's benchmark is likely just the beginning of figuring out how to teach machines not just what happens next, but why.
Usman Salis
