theNotebook

Entry 33 · Tuesday

embedding model migration: old vectors, new queries, no matches

3 November 2026

A woman holding a slip of paper studies shelves of books with strange symbols on their spines in a dim library at dusk, while plainer shelves stand across the aisle.
Mixed index versus parallel indexnew querymixed indexquiet missesnew queryparallel indexswitch when full

Kitchen table

It is like relabelling half the shelves in a library in a new language, so nobody finds the book they ask for.

Whiteboard

I am seeing teams swap embedding models and reindex in place. The new model scores better on the public retrieval benchmark. The rollout still breaks retrieval for days. Each embedding model draws its own private map of meaning. A query embedded by the new model lands on the new map. Half the index still sits on the old map. Similarity across two maps is noise that looks like a score. Nothing throws an error, and the top results just get quietly worse. The fix is a second index, not a mixed one. Build the new index beside the old one. Send each request to exactly one index, never to both. Switch over only when the backfill reaches every document. Tag each vector with its model version and refuse mixed reads. The trade-off is double storage and double write cost during the move. Measure recall on a fixed query set against both indexes. Also track the share of documents still on the old version. That share should reach zero before the switch, not after.

Boardroom

Upgrading the search model behind an AI assistant looks like a cheap win. Done in place, it quietly breaks answers for days or weeks. No alarm fires because nothing technically fails. Customers and staff simply get worse results and lose trust. The safe path runs old and new side by side, then switches at once. That costs extra storage and compute for the length of the move. It is a small price next to a silent drop in quality. The decision to take away is this. Does every model upgrade we ship have a tested switch and a rollback?

Tags

  • embeddings
  • vector search
  • rag
  • model migration
  • retrieval

All entries · 33 in all

Search the notebook