Spatial Intelligence in AI: Can an Agent Understand Its World?

Give an AI agent a map and ask it to find a door. A successful route leaves a question open: did the agent use the layout, or could it have followed a familiar sequence? We need another task to tell.

This is what interests me about spatial intelligence in AI: how information about a place becomes a useful decision. Can an agent use what it knows about an environment, carry that knowledge into a new situation, and change its mind when the world contradicts it?

I came to this through a broader question about context. Humans use expectations and beliefs to find our way through the world. We rarely stop to list everything that makes a doorway look usable or a warning seem credible. I want to understand what happens when we supply that kind of information to an AI, then expect it to act on rules whose implications seem obvious to us.

What would count as a useful world model?

Here, spatial reasoning means using relationships between places and objects to answer a question or choose an action. Which rooms connect? Can I reach that object? If this passage closes, is there another route?

A world model is a representation an agent can use to reason about an environment or predict the consequences of an action. It might preserve geometry, connectivity, object state, or rules about what changes when the agent acts. The evaluation needs to say which of these it tests.

The two terms describe different things: spatial reasoning is something an agent does; a world model is a representation it may use to do it. A memorized route can solve a navigation task. A model that predicts the next image may still struggle to answer whether an unvisited room is reachable.

That is why I would ask an agent several questions about the same world. Following one route tests something different from planning a new one, predicting a blocked passage, or explaining which observation would invalidate a plan.

There is useful recent research in this direction. Warrier and colleagues’ 2026 WorldTest paper evaluates whether learned models support different environment-level queries, including reachability and interventions. Its AutumnBench benchmark uses interactive grid worlds. The question is close to mine: how much of the environment can an agent use beyond what one successful trajectory reveals?

Visual environments introduce another difficulty: the agent has to assemble spatial information from observations over time. The CVPR 2025 paper Thinking in Space introduces VSI-Bench, with tasks involving distance, direction, and route planning from video. That makes a useful companion to a small symbolic world, where we can inspect the complete map directly.

Change the room, then ask again

Consider a proposed experiment with two exits. The agent receives a description saying that the blue door is locked and that the goal lies beyond the other exit. It chooses a route and reaches the goal.

I would then change one part of the setup at a time:

These are experiment designs, not results from a new benchmark. The comparisons would need matched instructions, observations, action rules, and opportunities to explore.

I would also include simpler strategies. A fixed action sequence tells us whether the task can be solved without adapting. Giving another agent the full map helps separate difficulties with observation from difficulties using the map. Neither comparison is useful unless we account for the information each agent received.

This is also why the representation deserves attention. A coordinate list, a connectivity graph, and a picture can describe the same place while making different relationships explicit. In The Map Is Not the Prompt, I explore that distinction. An experiment should make clear whether the agent inferred a map or received one with much of the reasoning already done.

Where belief enters the question

Belief is part of what drew me to this work. For humans, an expectation can guide an action long before we check whether it is true. What would it mean to study something similar in an AI without assuming that its experience resembles ours?

I would begin with behavior we can observe. Tell an agent that a marked tile restores energy. Does it repeat that statement to a successor? Does it seek the tile? After the tile fails to restore energy, does the agent keep trying it?

Those observations answer separate questions. Repetition shows that information survived a communication channel. A change in route may show that the information influenced action. A response to contradiction may show updating under that particular condition. Calling all three “belief” too early would hide the very differences I want to investigate. None alone establishes subjective, human-like belief.

Creation Garden asks how agents inherit descriptions of a simulated world. Its historical working paper is under revision after a measurement audit, and its earlier numerical claims have been withdrawn. The research question remains open: when a description passes between agents, what persists in the language, and what carries through into their choices?

The interface can change the result

An agent acts through software. Before interpreting a failed route, I want to know which action it requested and what the environment actually did with it. Did it walk into a wall, or did the interface reject a valid move? Was an API failure recorded as an action? Did the observation omit the door we expected it to use?

A final success score cannot resolve those questions. The record needs the input, requested action, resulting state, and any error between them. Otherwise we risk attributing a property of the harness to the agent.

The environment can also make a task easier than intended. A color might reveal the correct exit. An object name might encode the answer. A supplied graph might contain a route that a vision-based agent has to discover. Checking these details is part of evaluating spatial intelligence.

Google DeepMind’s work on interactive agents in video-game worlds offers another way to see why context matters. Its virtual playhouse lets people give agents situated instructions involving objects and actions. Even a request as ordinary as putting something close to something else depends on how the surrounding scene gives the words meaning.

That relationship between an instruction and the world around it is where I want Walking Stick Labs to work. A small world gives us somewhere to ask a precise question, inspect the answer, and decide what to change next.

I want to know when a supplied rule becomes useful to an agent, and which observations make it stop relying on that rule.

For a related method, Make the Model Commit looks at making a model’s claims explicit enough to check. The research projects show the settings in which I want to pursue these questions.