The bottom of the stack. Nothing above this makes sense without it. Titles are on the spines; the notes underneath are the three tellings, in short — for a ten-year-old, for the architect, and for the boardroom.
6 concepts on this shelf · Beginner → Intermediate · about 8 min read
Kitchen table — Imagine teaching someone to spot cats without ever writing down what a cat is. You just show them thousands of pictures — cat, not cat, cat, not cat — and one day they can do it themselves, even with cats they have never seen. That is the whole trick: instead of writing rules, you show examples and let the machine work out the rule. The catch is that it only knows what the pictures taught it. Show it nothing but house cats and its first tiger will go very badly.
Whiteboard — Fit a function to data: supervised when the labels exist, unsupervised when you are hunting for structure without them, self-supervised when the data is made to label itself. Everything downstream rests on one quiet assumption — that the distribution at inference resembles the distribution you trained on. Most production failures are not exotic; they are that assumption breaking while nobody was measuring it. The craft is less about picking the cleverest model and more about knowing what your data can and cannot testify to.
Boardroom — The system learns your patterns from your history, which means the quality of every answer is capped by the quality of the record you feed it. It does not know your policy; it knows what your organisation actually did, including the habits you would rather it had not learned. Budget for the data work first — cleaning, labelling, and keeping it current — because that, not the algorithm, is usually the constraint. And ask what happens when the world shifts: a model trained on last year is a photograph, not a camera.
machine learning, supervised learning, unsupervised learning, training data, model fitting
Kitchen table — Picture rows and rows of tiny switches, each one connected to the row behind it. One switch on its own is useless — it just passes a little signal along. But the first row notices edges, the next row notices shapes, the row after that notices whiskers, and somewhere near the end a switch lights up for 'cat'. Nobody told any switch what a whisker was. They worked it out together, by adjusting how strongly each one listens to its neighbours.
Whiteboard — Layers of weighted sums pushed through non-linearities. Depth buys you composition — features built out of features — while width buys raw capacity, and backpropagation is the bookkeeping that shares the blame for an error backwards through every weight that contributed to it. The perpetual surprise is how far this one recipe scales; the perpetual tax is that what any individual weight means is nobody's business, including yours. Interpretability is a research field precisely because the thing works better than we can explain.
Boardroom — This is the engine under nearly everything sold as AI today — vision, speech, language, all of it. The economics run one way: expensive to train, comparatively cheap to run, which is why a handful of labs train the giants and everyone else rents them. The governance point to hold onto is that a neural network cannot show its working the way a spreadsheet can. If a decision must be explainable to a regulator line by line, plan for that from day one or use something simpler.
neural network, deep learning, layers, activation function, backpropagation
Kitchen table — You are standing on a huge hill in thick fog, and you want to reach the valley. You cannot see the bottom — you cannot see anything — but you can feel the slope under your feet. So you take one small step downhill. Then you feel again, and step again. Do that a few million times and you end up somewhere low, even though you never once saw where you were going. That is how a machine learns: it feels which way 'less wrong' is, and takes a tiny step that way, over and over.
Whiteboard — Differentiate the loss with respect to every parameter, then step against the gradient. Everything else is commentary: batch sizes trade noise for speed, momentum smooths the ride, Adam rescales each dimension's steps, and the learning-rate schedule is where theory quietly gives way to folklore. The fog matters — the loss surface is high-dimensional and non-convex, and the honest answer for why we still find good minima is that overparameterised networks make most valleys good enough. Nobody fully knows; it works anyway.
Boardroom — This is literally what 'training' means: millions of small corrections, each one nudging the system slightly less wrong. It is why training takes weeks and megawatts, why progress shows up as a curve that flattens, and why 'just train it a bit more' eventually stops buying anything. When a team asks for more compute, they are asking for more steps down this hill. The strategic question is always whether the remaining descent is worth the electricity — and increasingly, someone has already walked most of the way down for you.
gradient descent, optimization, learning rate, slope, derivative, SGD, Adam
Kitchen table — Every time the machine guesses, it gets a score for how wrong it was — like golf, where a lower number is better. The whole game of learning is making that wrongness number smaller. But here is the sneaky part: the machine only cares about the score you chose. If you score it on 'how fast did you answer' it will get fast and sloppy. If you score it on 'never miss a cat' it will start calling dogs cats, just in case. You get exactly what you measure, nothing else.
Whiteboard — MSE for regression, cross-entropy for classification, and a long tail of task-specific objectives — but the design decision that matters is that the loss is the only definition of 'good' the model will ever see. Class imbalance, asymmetric costs, calibration: if it is not in the loss or the data, the optimiser will cheerfully trade it away. Proxy objectives are where systems go quietly wrong, because the model optimises the proxy with a literal-mindedness no human contract lawyer could match.
Boardroom — Whatever you choose to measure is what the system will optimise — including the things you did not mean to reward. Pick 'calls closed per hour' and watch quality fall; pick 'engagement' and watch outrage rise. This is not a hypothetical, it is the central operating risk of automated decision-making, and it deserves the same scrutiny you would give an incentive plan for people. Before signing off on any AI objective, ask the team: what is the cheapest ugly way to make this number go up? The model will find it.
loss function, mean squared error, MSE, cross entropy, objective function, error
Kitchen table — Computers only understand numbers, so everything else has to be turned into numbers first. 'Math', 'Science' and 'Art' cannot go in as words — but if you call them 1, 2 and 3, the computer thinks Art is somehow three times Math, which is nonsense. So instead you give each subject its own switch: Art is 'off, off, on'. A lot of the real work in machine learning is exactly this — turning messy real-world stuff into numbers without accidentally lying.
Whiteboard — One-hot, ordinal, target and hashing encoders; embeddings when the vocabulary explodes; multi-label columns that refuse to be tidy; and leakage — the silent killer — where information from the future or the label sneaks into a feature and the model aces validation while learning nothing. Feature work still beats model choice more often than anyone at a conference will admit. The unglamorous questions — what was known at prediction time, what is stable across seasons — are the ones that decide whether the thing survives production.
Boardroom — Most of the effort in a real AI project is not the model — it is getting messy business data into a shape a model can use, and that ratio has stayed stubbornly around four-to-one for a decade. When a project slips, this is usually where it slipped. It is also where institutional knowledge lives: the analyst who knows that the 'region' field changed meaning in 2021 is worth more to the project than another data scientist. Fund the plumbing and keep the people who understand the data's history.
feature engineering, one-hot encoding, categorical variables, ordinal encoding, multi-label, preprocessing
Kitchen table — Before you trust the machine, you give it a test — questions you already know the answers to — and count how many it gets right. But you have to be careful about which questions. If you quiz it on the same examples it studied, it just remembers the answers, like peeking at the answer sheet. So you always keep back some questions it has never seen. And one number is never enough: a doctor-machine that says 'nobody is sick' gets 99% right if almost nobody is sick, and is still completely useless.
Whiteboard — Hold-out sets and cross-validation for honesty; precision against recall because they pull in opposite directions and the right trade depends on the cost of each mistake; calibration because a confident wrong answer is worse than a hesitant one. Class imbalance turns accuracy into a vanity metric, and distribution shift turns last quarter's beautiful eval into fiction. The discipline is keeping a test set the training process has never touched — and treating any metric the team optimises directly as already compromised.
Boardroom — Never accept a single number. '92% accurate' means nothing until you know what guessing would have scored, what the errors cost, and who bears them — a false alarm and a missed case are rarely the same price. Ask for the metric, the baseline, and the confusion between the two kinds of wrong. And insist evaluation continues after launch: a model is measured on the day it shipped and then the world starts moving. The quiet degradation eighteen months in is the failure mode nobody budgets for.
model evaluation, accuracy, precision, recall, F1, ROC AUC, cross validation, baseline