Topic
Cost, latency and serving
Inference, caching, quantisation and model upgrades. What it takes to run AI in production.
Kitchen table
Every answer from a robot costs a little money and a little time. Some tricks make it cheaper and faster. These pages show which tricks really work.
Whiteboard
Serving is where cost and latency meet correctness. Caches, quantisation and speculative decoding all trade something. These entries show what each one costs you.
Boardroom
Running AI costs money on every call. Savings that break answers are not savings. These pages show where the real trade-offs sit.
Start with the concepts
-
Cost, Latency & ROI Route easy traffic to small models and hard cases up the tier.
-
MLOps & Deployment Versioning for models, data and prompts together – a prompt change is a deploy.
-
Data Pipelines Batch and streaming, orchestrators and retries, schema drift, late-arriving data, backfills that rewrite history, and idempotency so a rerun does not…
From the daily page
-
llm upgrade testing: the new model passed the old tests
2 November 2026 -
the questions looked alike and the answers did not
29 October 2026 -
the answers survived the squeeze and the reasons did not
26 October 2026 -
it deferred every ticket and called that caution
25 October 2026 -
the speed-up went backwards once we asked for json
23 October 2026 -
pinned to zero and still changing its mind
19 October 2026 -
amortising the housekeeping made the recall worse, not cheaper
18 October 2026 -
the cache turned out to be slower than thinking about it again
16 October 2026 -
the copy came out steadier than the thing it copied
15 October 2026 -
the thinking-off switch turned out to be a suggestion
10 October 2026 -
the pass rate went up and so did everything else
2 October 2026