Entry 30 · Saturday
the numbers were right and the units were not
Kitchen table
If a recipe says one spoon of salt and you read it as one cup, the maths is fine and dinner is ruined.
Whiteboard
Extraction pipelines get graded on the number and not the dimension. A model reads a table and returns 4.5 for capacity. The source said megawatts. The schema expected kilowatts. The field validates, the type checks, and the error is thousandfold. I have watched this survive three review layers at large US utilities. The cause is usually the prompt. We ask for a value and name the unit once, far from the field. The model treats the unit as context and not as a constraint. The trade-off is that strict unit fields raise refusal rates on messy documents. That is the better failure. Measure unit accuracy per field, not accuracy per row. Add a dimensional check that recomputes a derived quantity from the extracted parts. If load divided by voltage does not resemble a current, the row is wrong. Log the source span for every number. A number without its span cannot be audited later.
Boardroom
Our document extraction looks accurate because we grade the digits. We do not grade the units. A factor of a thousand passes every check we run today. In regulated filings and capital plans, that error carries real money and real scrutiny. The fix is cheap and it is not a model upgrade. It is a dimensional check and a stored source reference for every extracted number. The cost is more rows sent to a person for review. I would rather pay for review than defend a decimal place to a regulator. The question for this room is simple. Do we require a source citation for every number that reaches a filing?
Tags
All entries · 32 in all