the rehearsal went well because nobody else was in it

A benchmark out last week turned language-model agents loose on 119 chess endgames where a forced mate exists, and handed them a console with three separate verbs: make a real move, ask about the board, or simulate ahead. Across 8,568 recorded games, the simulate verb is where it got interesting. Just under a third of the accepted rollouts ran on until they hit an illegal move, and in 49 per cent of comparable cases the real defender replied with something other than the line the agent had just talked itself through. Naming the right first move, which is all a static benchmark asks for, happened in 26 per cent of trials. Only 14 per cent of those became a win. The pattern travels well beyond chess: a dry run checks whether the plan is well-formed, never whether the world will agree with it. So which of your agent's tools are rehearsals, and which are the thing itself?