Microsoft Research introduced MindTopo on August 12 to test whether vision-language models can track spatial relationships as a scene changes. The benchmark covers connectivity, separation, order, enclosure, and knots. A model might identify whether a path is open or whether sheep are inside a fence, then face a task where its own actions change the scene.
That second part exposes a weakness that still images can hide. In Microsoft's tests, multimodal models generally performed better when answering questions about a fixed scene than when planning actions over several steps. A model could recognize the layout but later choose a move that broke a required connection or violated the simulated environment's rules. Results for both kinds of task remained below observed human performance.
The benchmark uses controlled scenes, including mazes, pipes, fences, and ropes, so researchers can distinguish mistakes in visual recognition from mistakes made while acting. Adding generated images or video did not reliably solve the problem: simulated sequences could change important structural relationships.
MindTopo measures spatial reasoning in controlled simulations. It does not test OCR accuracy or establish that an assistant can reliably navigate a real interface.