#LLM(17)

July 2026
#React #LLM #Frontend #Markdown #AI

Streaming LLM Markdown in React Without Flicker

LLM tokens arrive one at a time, and re-parsing Markdown on every token flickers and drags. How to render streaming Markdown in React without the jank.

Read more →
July 2026
#LLM #FastAPI #Python #Backend #AI

Handling LLM API Rate Limits: Retries and Backoff

Your LLM backend returns 429s the moment traffic bursts. How to retry with backoff and jitter, respect Retry-After, and pace fan-out to stay under the limit.

Read more →
July 2026
#FastAPI #Python #AsyncIO #Backend #LLM

FastAPI Event Loop Blocking: Sync vs Async

One blocking call in a FastAPI route stalls every other request, including live SSE streams. Here is how the event loop breaks, and how to keep it free.

Read more →
July 2026
#AI #LLM #RAG #VectorSearch #Python

How HNSW Vector Search Actually Works

Every RAG stack leans on HNSW but treats it as a black box. Here is how the layered graph index finds nearest neighbors fast, and the knobs that matter.

Read more →
July 2026
#AI #LLM #RAG #Python #FastAPI

Cross-Encoder Reranking for RAG Pipelines

Retrieval puts the right chunk at rank 8, but the generator only reads the top few. How a cross-encoder reranker reorders RAG candidates, and where it fails.

Read more →
July 2026
#AI #LLM #RAG #Python #FastAPI

Chunking Strategies for RAG Pipelines

How to chunk documents for a RAG pipeline: why fixed-size splitting fails, structure-aware splitting, size and overlap tradeoffs, and the failure modes.

Read more →
July 2026
#AI #LLM #Agents #Python #Security

Running LLM-Generated Code in a Sandbox

An LLM that writes and runs code needs real isolation, not a try/except. How to sandbox AI-generated code with E2B microVMs, and the failure modes.

Read more →
July 2026
#AI #LLM #RAG #Evaluation #Python

Measuring RAG Retrieval Quality: recall@k, MRR

Changed your embeddings or added a reranker? Measure it. Build a golden set and score retrieval with recall@k, MRR, and nDCG before you trust the change.

Read more →
July 2026
#AI #LLM #RAG #OpenSearch #Python

Hybrid Search for RAG: BM25 + Vectors

Vector search alone misses exact IDs and error codes. Here's how to combine BM25 keyword search with dense retrieval, fuse the rankings with RRF, and rerank.

Read more →
June 2026
#AI #LLM #Agents #Python #Claude

How an LLM Agent Tool-Calling Loop Works

A practical look at the agent loop behind LLM tools: how the model asks to call a tool, your code runs it, and the result feeds back until the answer is done.

Read more →
June 2026
#AI #LLM #FastAPI #React #Python

Streaming LLM Responses from FastAPI with SSE

Stream LLM output token by token from a FastAPI backend with Server-Sent Events: working code, the EventSource client, proxy buffering, and failure modes.

Read more →
June 2026
#AI #LLM #RAG #FastAPI #Python

Incremental Indexing for RAG Pipelines

How to keep a RAG index fresh without full rebuilds: detect changed files with checksums, re-embed only what changed, and handle deletions safely.

Read more →