This paper asks a simple question: can modern 4D foundation models remember what they have seen? By 4D the authors mean three-dimensional scenes observed over time. They introduce a new benchmark called PersistBench to test whether models keep track of objects that move out of the camera’s view.

PersistBench is a dataset and a set of metrics built from 360° videos. The authors treat those full-view videos as an “omniscient” ground truth that records the whole scene, not just what a single camera sees at one moment. They use this ground truth to measure three object-centered abilities: object permanence (does the model remember that an object still exists and where it is?), motion continuity (does the model predict consistent motion over time?), and appearance preservation (does the object keep a consistent look when it reappears?).

Existing benchmarks typically use pixel-level scores that compare images directly. Those scores miss object-level memory because they do not provide a reference for objects once they leave the field of view. PersistBench aims to fill that gap by giving models a way to be checked against the full scene when objects are temporarily out of sight.

The authors evaluated a range of current models, including camera-controllable video models and 4D reconstruction models, across diverse categories in their dataset. They report that models can keep short-term consistency while objects stay nearby, but performance drops notably once objects leave the camera’s view. Their summary: “seeing is not remembering” — perception and short-term reconstruction do not imply robust long-term visual memory.

This matters because remembering objects is important for navigation, planning, and interaction in real-world tasks. If a model cannot reliably recall where an object went or how it looked when it left view, it will struggle in applications that require sustained awareness of a changing scene.