The Institute of Foundation Models at MBZUAI

PAN 2 World Model

Institute of Foundation Models

An open foundation world model built for physical intelligence.

A language model predicts the next word. PAN 2 predicts the next world. Built on generative latent prediction (GLP), it lets people and AI agents see what would happen before they act.

People cannot learn how to swim by reading a book. You have to basically jump into the water and start to feel it

Eric P. Xing Founder of IFM, President and University Professor of MBZUAI

A simulator where AI agents and humans can experiment before “real-world acting”

PAN 2 couples conceptual and perceptual reasoning. Its conceptual reasoning backbone, a multimodal LLM, works out what happens and when; its perceptual reasoning backbone, a 29B latent diffusion model trained from scratch, fills in how it looks and moves. Together, they let agents and people see the outcomes of candidate actions before choosing among them.

  • 01 Analyse and plan

    Accelerate mangrove forest growth to protect the coast

  • 02 Simulate

  • 03 Decide

    Mangrove ecosystem

Built for physical
intelligence, PAN 2
is grounded in time,
physics and space

Decide when to act, and see what follows

Time · Precise control over timed actions

Timing decides outcomes: a surfer has to pop up just as the wave lifts the board, and too early or too late, the ride is lost. PAN 2's conceptual reasoner writes out who does what and when, so actions can take effect in the order given. With PAN 2, you decide when each action happens. Change the timing or the order, and you see a different outcome, with each step building on the one before. Only a simulation faithful across the full plan can reliably guide an agent's decisions.

All videos generated by PAN 2.

Simulating physical reality, not painting pixels

Physics · Outcomes that follow physical laws

Pour milk, ink and oil into water: milk clouds it, ink spreads through it, and oil floats on top. An agent needs a simulation that obeys such physical laws and constraints, not just a visually convincing clip. PAN 2's perceptual backbone fills in what the conceptual backbone leaves open: geometry, appearance and the details of motion. Trained against real footage, it learns how the physical world evolves. It answers “what if?”, even for combinations of conditions absent from the training data.

All videos generated by PAN 2.

Change the viewpoint, not the world

Space · Spatial coherence under interaction

Turn the camera full circle, and the room you return to is the one you left. An agent's view changes with every move it makes, but the world it acts in should not. PAN 2's conceptual reasoning backbone writes camera motion into each next state, separately from what people and objects do, so the camera can move without changing the world it shows. What leaves the view stays in the world, keeping its identity and state until it comes back.

All videos generated by PAN 2.

Recent papers

IFM is developing AI as a shared resource for everyone, globally

  • Sunnyvale
  • Paris
  • Abu Dhabi

If that's the work you want to do, join us. Contact us