PAN 2 World Model
Institute of Foundation Models
An open foundation world model built for physical intelligence.
A language model predicts the next word. PAN 2 predicts the next world. Built on generative latent prediction (GLP), it lets people and AI agents see what would happen before they act.
- Press Release
- Technical Blog
- Research Paper Coming soon
People cannot learn how to swim by reading a book. You have to basically jump into the water and start to feel it
Eric P. Xing
Founder of IFM, President and University Professor of MBZUAI
A simulator where AI agents and humans can experiment before “real-world acting”
PAN 2 couples conceptual and perceptual reasoning. Its conceptual reasoning backbone, a multimodal LLM, works out what happens and when; its perceptual reasoning backbone, a 29B latent diffusion model trained from scratch, fills in how it looks and moves. Together, they let agents and people see the outcomes of candidate actions before choosing among them.
-
01 Analyse and plan
Accelerate mangrove forest growth to protect the coast
-
02 Simulate
-
03 Decide
Mangrove ecosystem
Built for physical
intelligence,
PAN 2
is grounded in time,
physics and space
Decide when to act, and see what follows
Time · Precise control over timed actions
Timing decides outcomes: a surfer has to pop up just as the wave lifts the board, and too early or too late, the ride is lost. PAN 2's conceptual reasoner writes out who does what and when, so actions can take effect in the order given. With PAN 2, you decide when each action happens. Change the timing or the order, and you see a different outcome, with each step building on the one before. Only a simulation faithful across the full plan can reliably guide an agent's decisions.
All videos generated by PAN 2.
Simulating physical reality, not painting pixels
Physics · Outcomes that follow physical laws
Pour milk, ink and oil into water: milk clouds it, ink spreads through it, and oil floats on top. An agent needs a simulation that obeys such physical laws and constraints, not just a visually convincing clip. PAN 2's perceptual backbone fills in what the conceptual backbone leaves open: geometry, appearance and the details of motion. Trained against real footage, it learns how the physical world evolves. It answers “what if?”, even for combinations of conditions absent from the training data.
All videos generated by PAN 2.
Change the viewpoint, not the world
Space · Spatial coherence under interaction
Turn the camera full circle, and the room you return to is the one you left. An agent's view changes with every move it makes, but the world it acts in should not. PAN 2's conceptual reasoning backbone writes camera motion into each next state, separately from what people and objects do, so the camera can move without changing the world it shows. What leaves the view stays in the world, keeping its identity and state until it comes back.
All videos generated by PAN 2.
Get started
-
Open weight
Download the model weights from Hugging Face.
- Hugging Face
-
Training code
Inspect, finetune, or train your own model from scratch.
- GitHub
-
Inference code
Run PAN 2 with vLLM, HF Diffusers or FastVideo.
Recent papers
-
Critique of World Model
Eric P. Xing, Mingkai Deng, Jinyu Hou
-
Critique of Agent Model
Eric P. Xing, Mingkai Deng, Jinyu Hou
-
WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning
Yuheng Zha, Yilei Wang, Qiyue Gao, Junrong Chen, Yujia Wu, Zhengfeng Lai, Zhengzhong Liu, Eric P. Xing
-
Efficient Agentic Reasoning Through Self-Regulated Simulative Planning
Mingkai Deng, Jinyu Hou, Lara Sá Neves, Varad Pimpalkhute, Taylor W. Killian, Zhengzhong Liu, Eric P. Xing
-
World Reasoning Arena
PAN Team, IFM: Qiyue Gao, Kun Zhou, Jiannan Xiang, Zihan Liu, et al.
-
PAN: A World Model for General, Actionable, and Long-Horizon World Simulation
PAN Team, IFM: Zihan Liu, Yi Gu, Mingkai Deng, Guangyi Liu, Zeyu Feng, Qiyue Gao, Yiyan Hu, Benhao Huang, Yichi Yang, Kun Zhou, Jiannan Xiang, Zhiting Hu, Zhengzhong Liu, Eric P. Xing
-
General Agentic Planning Through Simulative Reasoning with World Models
Mingkai Deng, Jinyu Hou, Zhiting Hu, Eric P. Xing
IFM is developing AI as a shared resource for everyone, globally
- Sunnyvale
- Paris
- Abu Dhabi
If that's the work you want to do, join us. Contact us