Entry 13 · Wednesday
the skill that won the argument wasn't the one you installed
Kitchen table
If two instruction cards on the shelf both explain how to tidy a room and your helper grabs whichever one is nearest, the room still gets tidy, but the rule you wrote about not touching the bookcase was on the card they didn't pick.
Whiteboard
A study out this week went looking in the gap between installing an agent skill and the agent actually using it. Across 312 confirmed pairs and 6,368 runs, roughly one installed skill in four sits alongside another claiming the same job, and the model picks between them on name and description alone. About one run in five quietly went to the neighbour. Tasks still passed, which is the whole problem: runs that opened the other skill first shed more than a third of the core functions only the installed one provided, including, in one case, a rule forbidding changes to git. Install location decided the winner, listing order barely mattered, and the final reply named the skill it had actually used in 0.9 per cent of substituted runs. The conflict is settled at the first read, before a single file is edited. So which instructions is your agent following, and where would you go to find out?
Boardroom
No boardroom telling for this one yet. Here is the whiteboard one.
A study out this week went looking in the gap between installing an agent skill and the agent actually using it. Across 312 confirmed pairs and 6,368 runs, roughly one installed skill in four sits alongside another claiming the same job, and the model picks between them on name and description alone. About one run in five quietly went to the neighbour. Tasks still passed, which is the whole problem: runs that opened the other skill first shed more than a third of the core functions only the installed one provided, including, in one case, a rule forbidding changes to git. Install location decided the winner, listing order barely mattered, and the final reply named the skill it had actually used in 0.9 per cent of substituted runs. The conflict is settled at the first read, before a single file is edited. So which instructions is your agent following, and where would you go to find out?
Tags
All entries · 32 in all