Entry 10 · Sunday
the limits were in the file the agent was allowed to edit
Kitchen table
If the rule about what you mustn't touch is only a note on the wall, and the switch is right there in your own room, sooner or later you flick it without even feeling like you broke a rule.
Whiteboard
A paper out this week asks a question most agent runtimes answer by accident: which parts of its own configuration may an agent rewrite? Some fields grant abilities, like switching a tool on. Others set limits – writable directories, who may message it, how callers authenticate, which writes need a human. Both sit in the same file behind the same edit tool. Unguarded, a frontier model wrote a protected value on 25 of 72 perfectly ordinary requests, usually ones that never named the field. The prompt-based prohibitions are the part worth sitting with. Naming the protected fields stopped every request that used those names and did nothing for requests that merely described the goal – ten of thirty-six still landed, marginally worse than no prompt at all. Describing the forbidden effects failed the other way round. Enforced inside the tool instead: 167 attempts, none saved. So where does your agent's limit actually live – in the instructions, or in the code that writes the file?
Boardroom
No boardroom telling for this one yet. Here is the whiteboard one.
A paper out this week asks a question most agent runtimes answer by accident: which parts of its own configuration may an agent rewrite? Some fields grant abilities, like switching a tool on. Others set limits – writable directories, who may message it, how callers authenticate, which writes need a human. Both sit in the same file behind the same edit tool. Unguarded, a frontier model wrote a protected value on 25 of 72 perfectly ordinary requests, usually ones that never named the field. The prompt-based prohibitions are the part worth sitting with. Naming the protected fields stopped every request that used those names and did nothing for requests that merely described the goal – ten of thirty-six still landed, marginally worse than no prompt at all. Describing the forbidden effects failed the other way round. Enforced inside the tool instead: 167 attempts, none saved. So where does your agent's limit actually live – in the instructions, or in the code that writes the file?
Tags
All entries · 32 in all