Interestingly, we can even analyze the generated image sequences using traditional computer vision algorithms to see the specific trajectories models take during in-context physical simulation. Like chain-of-thought, these intermediate steps are often structured and interpretable.
Excited to have this out! This was a fun project that started with YingQiao and I discussing whether VLMs can do mental simulation of physics like people do, and it culminated in a new method where we prompted image generation models to simulate a series of images frame-by-frame.
out now in Open Mind: "People Evaluate Agents Based on the Algorithms That Drive Their Behavior"
by Bigelow & me
Paper: direct.mit.edu/opmi/article...
OSF: osf.io/yzbrq/?view_...
Our latest on the cognitive science of LLMs! To be presented @CogSci2025 🎉
LLMs are increasingly involved in human collaborations. How do LLMs assign responsibility and reward to collaborators? Is it similar to how humans do it? 🤖🧑
📃 gershmanlab.com/pubs/XiangBi... (1/4)
