llm api rate limits: the retries made the outage longer

I watched a model provider slow down for a few minutes. Our services saw timeouts and rate limit errors. Every caller retried at once, on the same fixed delay. Agents made it worse, because one task fans out into many calls. Each failed tool call retried, and then the planner retried the whole step. Retries multiplied across layers, so load rose as capacity fell. The provider stayed saturated long after its fault cleared. The fix is boring, and it works. Use exponential backoff with full jitter. Honour the Retry-After header when the server sends one. Keep a retry budget, so retries stay a small share of traffic. Retry in one layer only, and let the other layers fail fast. Add a circuit breaker that sheds low-value work first. The trade-off is real. Some requests fail that might have passed. I measure retries per original request, at every layer. I also watch time to recover after an incident, not only the error rate. When retries outnumber first attempts, the system is attacking itself.