Correlation IDs in FastAPI Logs
One async request scatters a dozen log lines through concurrent traffic. Thread a correlation ID through FastAPI with contextvars and trace it in OpenSearch.
#FastAPI(35)
Read more →
LLM Model Routing: Send Easy Prompts to Small Models
Most prompts hitting an LLM app are simple. Route those to a small model and escalate only the hard ones to a large model to cut cost and keep quality.
FastAPI Request Validation with Pydantic v2
FastAPI validates request bodies with Pydantic v2 before your handler runs. Models, Field constraints, custom validators, the 422 response, and failure modes.
RAG Citations: Make the LLM Cite Its Sources
How to make a RAG system cite its sources: force the model to reference chunks by ID, then verify each citation against the source text before trusting it.
Agentic RAG: When Should It Retrieve?
Naive RAG retrieves on every turn, even when it shouldn't. How agentic RAG lets the model route, grade its own context, and re-retrieve, plus the tradeoffs.
FastAPI WebSockets for Streaming LLM Chat
SSE streams one direction only. When an LLM chat needs the browser to interrupt a running response, FastAPI WebSockets give you a full-duplex channel.
Parsing PDFs for RAG: Text, Tables, and OCR
How to parse PDFs for a RAG pipeline: detect born-digital vs scanned files, pull out tables without flattening them, and reach for OCR only when you must.
PodDisruptionBudgets: Survive a Node Drain
A cluster upgrade drains a node and evicts every replica of your service at once. How a PodDisruptionBudget caps voluntary disruptions and keeps you online.
Idempotency Keys: Safe API Retries in FastAPI
A network retry can submit the same request twice. How idempotency keys in FastAPI make POST endpoints safe to retry without duplicate side effects.
Prompt Caching with the Claude API
Prompt caching reuses a stable prompt prefix to cut Claude API cost and latency. How breakpoints, TTLs, and cache reads work, and what quietly breaks a hit.
Circuit Breakers for LLM API Calls
When an LLM provider degrades, retries make it worse. A practical guide to adding a circuit breaker in Python: the three states, tuning, and failure modes.
Autoscaling a FastAPI Service with the Kubernetes HPA
The Kubernetes HPA scales pods from a metric, but its defaults thrash and lag under real load. How the control loop works, how to tune it, and where it breaks.
Tracing a RAG Pipeline with OpenTelemetry
Trace a RAG pipeline with OpenTelemetry: instrument FastAPI, put each retrieval and model step in its own span, and read the latency waterfall in OpenSearch.
Fan Out Concurrent LLM Calls with asyncio.gather
Awaiting retrieval and LLM calls one by one wastes seconds per request. Here's how to fan them out with asyncio.gather, bound it, and handle partial failures.
Build a Semantic Cache for LLM Apps
Semantic caching for LLM apps: cache answers by embedding similarity in FastAPI, tune the cutoff, and avoid false cache hits. Working code and failure modes.
Rate Limiting a FastAPI Service with a Token Bucket
Add per-user rate limiting to a FastAPI backend with the token bucket algorithm: an in-process version, an atomic Redis script, 429s, and the failure modes.
Async Connection Pooling in FastAPI
Your async FastAPI app stalls under load, then times out. Here is how SQLAlchemy pool_size and max_overflow really work, and how to size them for Postgres.
How FastAPI Dependency Injection Actually Works
A practical guide to FastAPI dependency injection: how Depends resolves a graph, yield setup and teardown, per-request caching, and where it leaks.
FastAPI BackgroundTasks vs a Task Queue
FastAPI BackgroundTasks run inside your web process and disappear on restart. When that is fine, when you need a real task queue, and how to move over.
RAG Vector Search in Postgres with pgvector
Store and query RAG embeddings inside Postgres with pgvector: HNSW indexing, distance operators, metadata filtering, hybrid search, and the tradeoffs I hit.
Graceful Shutdown for FastAPI on Kubernetes
A rolling deploy sends SIGTERM and kills your FastAPI pod mid-request, dropping live SSE streams. How to catch it, drain connections, and shut down cleanly.
Prompt Caching: Cut LLM Cost and Latency
Prompt caching reuses a request's prefix to cut LLM cost and latency. How the prefix match works, where to put the breakpoint, and the silent cache misses.
Query Rewriting for Better RAG Retrieval
Short, vague, follow-up questions don't match how your docs are written. How query rewriting, multi-query expansion, and HyDE fix retrieval before it runs.
Getting Reliable JSON Out of an LLM
Getting an LLM to return JSON is easy; getting valid JSON every time is not. How to use JSON Schema, constrained decoding, and validation to make it reliable.
Handling LLM API Rate Limits: Retries and Backoff
Your LLM backend returns 429s the moment traffic bursts. How to retry with backoff and jitter, respect Retry-After, and pace fan-out to stay under the limit.
FastAPI Event Loop Blocking: Sync vs Async
One blocking call in a FastAPI route stalls every other request, including live SSE streams. Here is how the event loop breaks, and how to keep it free.
Cross-Encoder Reranking for RAG Pipelines
Retrieval puts the right chunk at rank 8, but the generator only reads the top few. How a cross-encoder reranker reorders RAG candidates, and where it fails.
Chunking Strategies for RAG Pipelines
How to chunk documents for a RAG pipeline: why fixed-size splitting fails, structure-aware splitting, size and overlap tradeoffs, and the failure modes.
Streaming LLM Responses from FastAPI with SSE
Stream LLM output token by token from a FastAPI backend with Server-Sent Events: working code, the EventSource client, proxy buffering, and failure modes.
Incremental Indexing for RAG Pipelines
How to keep a RAG index fresh without full rebuilds: detect changed files with checksums, re-embed only what changed, and handle deletions safely.
CloudCanvasAI
AI-powered document creation platform pairing Claude with a live .docx / .pptx / .pdf / .xlsx preview. Chat on the left, documents render live on the right, each user gets their own isolated E2B sandbox.
Argusa AI Challenge 2025 · Runner-Up 🥈
Secured 2nd place at the Argusa AI Challenge 2025 by building ARGRAG, a RAG system for complex enterprise document corpora with multi-modal analysis, metadata intelligence, and real-time performance.
CMS Workflow Management DevOps
Developing, Operating and extending on WMCore + Unified, so CMS Monte Carlo and reconstruction workloads stay healthy across the WLCG sites.
Archi · LLM Copilot for CMS Ops
Partnering with MIT to build an LLM-powered RAG copilot for CMS computing operations.
Gemini Alchemy
A Gemini-powered tool that turns abstract ideas and complex concepts into visually appealing mindmaps. FastAPI backend, Streamlit frontend.