Retrieval and context

Retrieval (RAG) Chunk the documents, embed them, retrieve by hybrid search (dense vectors plus keyword – dense alone misses part numbers and names), rerank, then…

Tokens & Embeddings Subword tokenisation sets your context budget and your bill – token counts, not word counts, are the unit of everything.

Prompting & Context Instructions, examples, retrieved documents, tool results and conversation history all compete for one finite context window, and the model attends…

Large Language Models Transformers and attention, pretrained on next-token prediction at a scale that turns a party trick into reasoning-shaped behaviour, then aligned…

the shortlist decided the score before the reranker did

the questions looked alike and the answers did not

the retriever was fine and the pdf was not

it found the needle every time and still lost count

the summary kept the answer and dropped the hedge

amortising the housekeeping made the recall worse, not cheaper

the rule at the top of the query turned out to be more query

nobody asks the whole question on the first turn

we kept pruning the memories that never got a turn