Field Note · Craft
Reading the Failure Logs
What I actually look at when an agent fails.
Key Takeaway
Identify agent failures by analyzing transition signatures (such as tight loop cycles vs. high-entropy random walks) to construct precise regression fixes rather than vague general improvements.
When a long-horizon agent fails, the reward curve is the last thing I look at. The first is the transition signature. Random failure and trapped failure look nothing alike in the logs: one is high-entropy wandering across many states, the other is a tight cycle through the same two or three. The shape tells you which problem you have before any metric does.
Most of the work is naming the failure. Once a loop has a name — this room, these two connectors, this many repeats — it stops being “the agent is bad” and becomes a specific thing with a specific fix. Vague failure resists iteration. Named failure invites it.