theNotebook

Entry 18 · Monday

pinned to zero and still changing its mind

19 October 2026

Shelf two · The toolkit

Kitchen table

If you ask the same question twice and get two different answers, it might not be the computer changing its mind, it might just be the machine underneath adding things up in a slightly different order.

Whiteboard

Someone sent fourteen commercial model deployments the same request over and over, with every deterministic setting the interface offered pinned down, and all fourteen eventually disagreed with themselves. In twelve the verdict flipped, not just the wording. Turning the reasoning down didn't help, and the same open-weight model was reproducible or not depending purely on how it happened to be served. The unsettling part isn't the model. When they rebuilt the thing to be deterministic end to end, the wobble turned up in the OCR, the embeddings, the approximate index, the GPU model and the scheduler's batching, none of which anyone thinks of as the part that decides anything. We pin temperature to zero and call it reproducibility, when the sampler was only ever one of seven places a run can quietly diverge. So if your recorded configuration doesn't name the GPU and the serving flags, what is it a record of?

Boardroom

No boardroom telling for this one yet. Here is the whiteboard one.

Someone sent fourteen commercial model deployments the same request over and over, with every deterministic setting the interface offered pinned down, and all fourteen eventually disagreed with themselves. In twelve the verdict flipped, not just the wording. Turning the reasoning down didn't help, and the same open-weight model was reproducible or not depending purely on how it happened to be served. The unsettling part isn't the model. When they rebuilt the thing to be deterministic end to end, the wobble turned up in the OCR, the embeddings, the approximate index, the GPU model and the scheduler's batching, none of which anyone thinks of as the part that decides anything. We pin temperature to zero and call it reproducibility, when the sampler was only ever one of seven places a run can quietly diverge. So if your recorded configuration doesn't name the GPU and the serving flags, what is it a record of?

Tags

  • determinism
  • reproducibility
  • inference
  • serving
  • mlops

All entries · 32 in all

Search the notebook