Shelf one · How a machine learns
Neural Networks
Kitchen table
Picture rows and rows of tiny switches, each one connected to the row behind it. One switch on its own is useless. It just passes a little signal along. But the first row notices edges, the next row notices shapes, the row after that notices whiskers, and somewhere near the end a switch lights up for 'cat'. Nobody told any switch what a whisker was. They worked it out together, by adjusting how strongly each one listens to its neighbours.
Whiteboard
Layers of weighted sums pushed through non-linearities. Depth buys you composition, features built out of features, while width buys raw capacity, and backpropagation is the bookkeeping that shares the blame for an error backwards through every weight that contributed to it. The perpetual surprise is how far this one recipe scales. The perpetual tax is that what any individual weight means is nobody's business, including yours. Interpretability is a research field precisely because the thing works better than we can explain.
Boardroom
This is the engine under nearly everything sold as AI today – vision, speech, language, all of it. The economics run one way: expensive to train, comparatively cheap to run, which is why a handful of labs train the giants and everyone else rents them. The governance point to hold onto is that a neural network cannot show its working the way a spreadsheet can. If a decision must be explainable to a regulator line by line, plan for that from day one or use something simpler.
Related concepts