Reflection Is Not Control

There is a popular intuition, visible in chain-of-thought, self-critique, and reflexion-style loops: if an agent reasons out loud about its situation, it will act better. The narration is supposed to feed back into behaviour. I built an embodied agent with a first-person diary to test exactly that, and the result was humbling — the diary was a near-perfect observer and a near-useless controller.

A diary entry would read “I keep circling this shoreline, I should head inland,” while the policy took the same exit twenty-eight times in a row. The narration correctly diagnosed the trap. It did not open it. Feeding the diary back into the agent’s context did not produce a measurably different escape rate.

The language was descriptive, not causal. It tracked the situation faithfully and changed nothing about the next action.

Structured behavioural memory did the control work — loop counters, exit-failure tallies, no-op pressure. Cheap, unglamorous state. The eloquent channel described the cage; the dumb counters picked the lock. When the agent was genuinely stuck, it was the counters, not the prose, that broke the cycle.

Keep reflection — as instrumentation. A narrated trace of what the agent believed at each step is gold for debugging and interpretability, and it is the most human-legible artifact an agent produces. Just do not expect narration to become control unless you explicitly wire it to control. Self-talk is not a planner, and an agent that can describe its failure is not thereby an agent that can avoid it.