Circuit Breakers for LLM API Calls
When an LLM provider degrades, retries make it worse. A practical guide to adding a circuit breaker in Python: the three states, tuning, and failure modes.
#Backend(7)
Read more →
Rate Limiting a FastAPI Service with a Token Bucket
Add per-user rate limiting to a FastAPI backend with the token bucket algorithm: an in-process version, an atomic Redis script, 429s, and the failure modes.
Async Connection Pooling in FastAPI
Your async FastAPI app stalls under load, then times out. Here is how SQLAlchemy pool_size and max_overflow really work, and how to size them for Postgres.
How FastAPI Dependency Injection Actually Works
A practical guide to FastAPI dependency injection: how Depends resolves a graph, yield setup and teardown, per-request caching, and where it leaks.
FastAPI BackgroundTasks vs a Task Queue
FastAPI BackgroundTasks run inside your web process and disappear on restart. When that is fine, when you need a real task queue, and how to move over.
Handling LLM API Rate Limits: Retries and Backoff
Your LLM backend returns 429s the moment traffic bursts. How to retry with backoff and jitter, respect Retry-After, and pace fan-out to stay under the limit.
FastAPI Event Loop Blocking: Sync vs Async
One blocking call in a FastAPI route stalls every other request, including live SSE streams. Here is how the event loop breaks, and how to keep it free.