Entry 16 · Saturday
the guardrail passed because nobody tried the door
Kitchen table
You can't tell whether a smoke alarm works just because the house hasn't burned down. You only know once something actually sets it off.
Whiteboard
A reliability harness published this week quietly adds a third verdict to guardrail tests, next to pass and fail: not_exercised. It fires when the agent never attempted the forbidden action at all, so the control meant to stop it never got a turn, and calling that a pass would be a lie told in green. The telling detail is that both the happy-path scenarios and the deliberately nasty one came back not_exercised – those runs stayed safe because the agent happened to behave itself, not because the authorisation gateway did anything. Most safety suites are assembled out of golden paths, which means most of them are measuring restraint and filing it as enforcement, and a hundred per cent effectiveness rate over controls that never fired is a number about nothing. Worth asking, then: which of your guardrails have you actually watched say no?
Boardroom
No boardroom telling for this one yet. Here is the whiteboard one.
A reliability harness published this week quietly adds a third verdict to guardrail tests, next to pass and fail: not_exercised. It fires when the agent never attempted the forbidden action at all, so the control meant to stop it never got a turn, and calling that a pass would be a lie told in green. The telling detail is that both the happy-path scenarios and the deliberately nasty one came back not_exercised – those runs stayed safe because the agent happened to behave itself, not because the authorisation gateway did anything. Most safety suites are assembled out of golden paths, which means most of them are measuring restraint and filing it as enforcement, and a hundred per cent effectiveness rate over controls that never fired is a number about nothing. Worth asking, then: which of your guardrails have you actually watched say no?
Tags
All entries · 32 in all