#Python(55)

September 2026

#OpenSearch #Search #Data Structures #RAG #Python

How Roaring Bitmaps Make Filters Fast

Roaring bitmaps store integer sets in three container types so filters stay small and fast. How they work, why OpenSearch leans on them, and the tradeoffs.

September 2026

#FastAPI #Observability #OpenSearch #Python #Backend

Correlation IDs in FastAPI Logs

One async request scatters a dozen log lines through concurrent traffic. Thread a correlation ID through FastAPI with contextvars and trace it in OpenSearch.

September 2026

#RAG #LLM #Python #Search #Data Engineering

Dedupe a RAG Corpus with MinHash and LSH

Exact md5 hashing misses near-duplicate documents; comparing every pair is O(n²). How MinHash and LSH find and cluster them across a RAG corpus.

September 2026

#FastAPI #Pydantic #Python #API #Backend

FastAPI Request Validation with Pydantic v2

FastAPI validates request bodies with Pydantic v2 before your handler runs. Models, Field constraints, custom validators, the 422 response, and failure modes.

September 2026

#FastAPI #WebSockets #LLM #Python #React

FastAPI WebSockets for Streaming LLM Chat

SSE streams one direction only. When an LLM chat needs the browser to interrupt a running response, FastAPI WebSockets give you a full-duplex channel.

September 2026

#AI #LLM #RAG #Retrieval #Python

Maximal Marginal Relevance for RAG

Top-k vector search keeps returning near-duplicate chunks that fill the context window. How Maximal Marginal Relevance reranks for relevant, diverse results.

September 2026

#RAG #LLM #Retrieval #Embeddings #Python

Parent Document Retrieval: Search Small, Answer Big

Small chunks search precisely but read like fragments. Parent document retrieval matches on child chunks, then feeds the whole parent to the model.

September 2026

#RAG #LLM #Python #FastAPI #AI

Parsing PDFs for RAG: Text, Tables, and OCR

How to parse PDFs for a RAG pipeline: detect born-digital vs scanned files, pull out tables without flattening them, and reach for OCR only when you must.

September 2026

#FastAPI #Backend #Reliability #API #Python

Idempotency Keys: Safe API Retries in FastAPI

A network retry can submit the same request twice. How idempotency keys in FastAPI make POST endpoints safe to retry without duplicate side effects.

September 2026

#AI #RAG #OpenSearch #Search #Python

Reciprocal Rank Fusion for Hybrid Search

Reciprocal Rank Fusion merges two ranked lists without tuning score weights. The RRF formula, why the k constant matters, and how to run it in OpenSearch.

August 2026

#AI #LLM #GPU #Inference #Python

How Continuous Batching Speeds Up LLM Serving

A naive LLM server holds finished slots idle until the slowest request in the batch drains. Continuous batching refills them every step. How it works.

August 2026

#AI #LLM #Claude #FastAPI #Python

Prompt Caching with the Claude API

Prompt caching reuses a stable prompt prefix to cut Claude API cost and latency. How breakpoints, TTLs, and cache reads work, and what quietly breaks a hit.

August 2026

#LLM #FastAPI #Python #Reliability #Backend

Circuit Breakers for LLM API Calls

When an LLM provider degrades, retries make it worse. A practical guide to adding a circuit breaker in Python: the three states, tuning, and failure modes.

August 2026

#AI #LLM #GPU #Inference #Python

Prefill vs Decode: Two Phases of LLM Inference

The first token is slow, the rest stream fast. Why LLM inference splits into a compute-bound prefill and a memory-bound decode, and what it costs you.

August 2026

#AI #LLM #RAG #Retrieval #Python

HyDE: Hypothetical Document Embeddings for RAG

Vector search fails when a short question looks nothing like its answer. HyDE has an LLM draft a fake answer, embeds that, and retrieves against it instead.

August 2026

#AI #LLM #RAG #Evaluation #Python

LLM-as-a-Judge: Scoring RAG Answer Quality

Retrieval metrics say the right docs came back, not that the answer is right. Build an LLM-as-a-judge to score RAG answers for faithfulness and quality.

August 2026

#Python #AsyncIO #FastAPI #LLM #RAG

Fan Out Concurrent LLM Calls with asyncio.gather

Awaiting retrieval and LLM calls one by one wastes seconds per request. Here's how to fan them out with asyncio.gather, bound it, and handle partial failures.

August 2026

#AI #LLM #RAG #FastAPI #Python

Build a Semantic Cache for LLM Apps

Semantic caching for LLM apps: cache answers by embedding similarity in FastAPI, tune the cutoff, and avoid false cache hits. Working code and failure modes.

August 2026

#FastAPI #Python #Redis #Backend #LLM

Rate Limiting a FastAPI Service with a Token Bucket

Add per-user rate limiting to a FastAPI backend with the token bucket algorithm: an in-process version, an atomic Redis script, 429s, and the failure modes.

August 2026

#AI #LLM #GPU #Inference #Python

How FlashAttention Speeds Up the Attention Layer

Attention is memory-bound, not compute-bound. Here's how FlashAttention uses tiling and online softmax to skip the N×N matrix and run exact attention faster.

August 2026

#FastAPI #Python #PostgreSQL #SQLAlchemy #Backend

Async Connection Pooling in FastAPI

Your async FastAPI app stalls under load, then times out. Here is how SQLAlchemy pool_size and max_overflow really work, and how to size them for Postgres.

August 2026

#AI #LLM #Tokenization #NLP #Python

How Byte-Pair Encoding Tokenizes Text for LLMs

Byte-pair encoding turns text into the tokens an LLM bills and reasons over. How BPE merges are learned, why token counts drive cost, and where it breaks.

August 2026

#FastAPI #Python #Backend #AI #API

How FastAPI Dependency Injection Actually Works

A practical guide to FastAPI dependency injection: how Depends resolves a graph, yield setup and teardown, per-request caching, and where it leaks.

August 2026

#AI #LLM #RAG #VectorSearch #Python

Late Interaction Retrieval for RAG with ColBERT

A bi-encoder averages token detail away; a cross-encoder is too slow to rank a corpus. Late interaction with ColBERT sits between them. Here is how it works.

August 2026

#FastAPI #Python #Backend #Redis #Architecture

FastAPI BackgroundTasks vs a Task Queue

FastAPI BackgroundTasks run inside your web process and disappear on restart. When that is fine, when you need a real task queue, and how to move over.

August 2026

#AI #LLM #RAG #VectorSearch #Python

Metadata Filtering in Vector Search for RAG

Adding a metadata filter to a vector search can silently return fewer results or wreck recall. How post-filter, pre-filter, and filterable HNSW actually differ.

July 2026

#FastAPI #Kubernetes #Python #DevOps #LLM

Graceful Shutdown for FastAPI on Kubernetes

A rolling deploy sends SIGTERM and kills your FastAPI pod mid-request, dropping live SSE streams. How to catch it, drain connections, and shut down cleanly.

July 2026

#AI #LLM #Agents #Python #RAG

Managing the Context Window in Long Agent Runs

An LLM agent that runs long enough fills its context window and starts to slow or fail. How to prune, compact, and offload context so agents keep going.

July 2026

#AI #LLM #RAG #Python #FastAPI

Query Rewriting for Better RAG Retrieval

Short, vague, follow-up questions don't match how your docs are written. How query rewriting, multi-query expansion, and HyDE fix retrieval before it runs.

July 2026

#AI #LLM #Evaluation #RAG #Python

LLM-as-a-Judge: Evaluating LLM Output Quality

Human review does not scale for grading LLM answers. How to use an LLM as a judge: write a rubric, score with structured output, and control the biases.

July 2026

#AI #LLM #Caching #Embeddings #Python

How to Build a Semantic Cache for LLM Apps

Exact-match caching misses paraphrases, so LLM bills stay high. Here is how to build a semantic cache with embeddings, a similarity threshold, and its traps.

July 2026

#AI #LLM #Agents #MCP #Python

Build an MCP Server for Your LLM Agent

MCP standardizes how LLM agents reach your tools and data. A hands-on guide to building an MCP server in Python, picking a transport, and where it breaks.

July 2026

#AI #LLM #Structured Outputs #Python #FastAPI

Getting Reliable JSON Out of an LLM

Getting an LLM to return JSON is easy; getting valid JSON every time is not. How to use JSON Schema, constrained decoding, and validation to make it reliable.

July 2026

#AI #LLM #GPU #Inference #Python

LLM KV Cache: Why GPU Memory Runs Out

A long prompt or a long agent run can hit CUDA out of memory, and the KV cache is usually why. How it grows, the per-token math, and how to shrink it.

July 2026

#LLM #FastAPI #Python #Backend #AI

Handling LLM API Rate Limits: Retries and Backoff

Your LLM backend returns 429s the moment traffic bursts. How to retry with backoff and jitter, respect Retry-After, and pace fan-out to stay under the limit.

July 2026

#FastAPI #Python #AsyncIO #Backend #LLM

FastAPI Event Loop Blocking: Sync vs Async

One blocking call in a FastAPI route stalls every other request, including live SSE streams. Here is how the event loop breaks, and how to keep it free.

July 2026

#AI #LLM #RAG #VectorSearch #Python

How HNSW Vector Search Actually Works

Every RAG stack leans on HNSW but treats it as a black box. Here is how the layered graph index finds nearest neighbors fast, and the knobs that matter.

July 2026

#AI #LLM #RAG #Python #FastAPI

Cross-Encoder Reranking for RAG Pipelines

Retrieval puts the right chunk at rank 8, but the generator only reads the top few. How a cross-encoder reranker reorders RAG candidates, and where it fails.

July 2026

#AI #LLM #RAG #Python #FastAPI

Chunking Strategies for RAG Pipelines

How to chunk documents for a RAG pipeline: why fixed-size splitting fails, structure-aware splitting, size and overlap tradeoffs, and the failure modes.

July 2026

#AI #LLM #Agents #Python #Security

Running LLM-Generated Code in a Sandbox

An LLM that writes and runs code needs real isolation, not a try/except. How to sandbox AI-generated code with E2B microVMs, and the failure modes.

July 2026

#AI #LLM #RAG #Evaluation #Python

Measuring RAG Retrieval Quality: recall@k, MRR

Changed your embeddings or added a reranker? Measure it. Build a golden set and score retrieval with recall@k, MRR, and nDCG before you trust the change.

July 2026

#AI #LLM #RAG #OpenSearch #Python

Hybrid Search for RAG: BM25 + Vectors

Vector search alone misses exact IDs and error codes. Here's how to combine BM25 keyword search with dense retrieval, fuse the rankings with RRF, and rerank.

June 2026

#AI #LLM #FastAPI #React #Python

Streaming LLM Responses from FastAPI with SSE

Stream LLM output token by token from a FastAPI backend with Server-Sent Events: working code, the EventSource client, proxy buffering, and failure modes.

June 2026

#AI #LLM #Agents #Python #Claude

How an LLM Agent Tool-Calling Loop Works

A practical look at the agent loop behind LLM tools: how the model asks to call a tool, your code runs it, and the result feeds back until the answer is done.

June 2026

#AI #LLM #RAG #FastAPI #Python

Incremental Indexing for RAG Pipelines

How to keep a RAG index fresh without full rebuilds: detect changed files with checksums, re-embed only what changed, and handle deletions safely.

November 2025

#AI #LLM #Claude #RAG #FastAPI #React #E2B #Firebase #Python

CloudCanvasAI

AI-powered document creation platform pairing Claude with a live .docx / .pptx / .pdf / .xlsx preview. Chat on the left, documents render live on the right, each user gets their own isolated E2B sandbox.

November 2025

#AI #LLM #RAG #Multimodal AI #FastAPI #Python #Hackathon #EPFL #Argusa

Argusa AI Challenge 2025 · Runner-Up 🥈

Secured 2nd place at the Argusa AI Challenge 2025 by building ARGRAG, a RAG system for complex enterprise document corpora with multi-modal analysis, metadata intelligence, and real-time performance.

February 2024