How Roaring Bitmaps Make Filters Fast
Roaring bitmaps store integer sets in three container types so filters stay small and fast. How they work, why OpenSearch leans on them, and the tradeoffs.
#RAG(42)
LLM Model Routing: Send Easy Prompts to Small Models
Most prompts hitting an LLM app are simple. Route those to a small model and escalate only the hard ones to a large model to cut cost and keep quality.
Binary Quantization for Vector Search
Float32 embeddings make a vector index expensive to keep in RAM. Binary quantization cuts them 32x; Hamming search plus rescoring keeps recall high.
Dedupe a RAG Corpus with MinHash and LSH
Exact md5 hashing misses near-duplicate documents; comparing every pair is O(n²). How MinHash and LSH find and cluster them across a RAG corpus.
RAG Citations: Make the LLM Cite Its Sources
How to make a RAG system cite its sources: force the model to reference chunks by ID, then verify each citation against the source text before trusting it.
OpenSearch as a Vector Store for RAG
Run RAG retrieval inside the OpenSearch cluster you already operate. Set up a knn_vector index, pick faiss vs lucene, and avoid the memory traps.
Agentic RAG: When Should It Retrieve?
Naive RAG retrieves on every turn, even when it shouldn't. How agentic RAG lets the model route, grade its own context, and re-retrieve, plus the tradeoffs.
Cosine, Dot Product, or Euclidean for Embeddings?
Cosine, dot product, and Euclidean rank vector search results the same on normalized embeddings, differently otherwise. How to pick and configure one.
Maximal Marginal Relevance for RAG
Top-k vector search keeps returning near-duplicate chunks that fill the context window. How Maximal Marginal Relevance reranks for relevant, diverse results.
Parent Document Retrieval: Search Small, Answer Big
Small chunks search precisely but read like fragments. Parent document retrieval matches on child chunks, then feeds the whole parent to the model.
IVF Vector Search: Search Fewer Vectors
Brute-force vector search compares every embedding. How an IVF index partitions vectors into cells, probes only the nearest, and where the nprobe knob bites.
Parsing PDFs for RAG: Text, Tables, and OCR
How to parse PDFs for a RAG pipeline: detect born-digital vs scanned files, pull out tables without flattening them, and reach for OCR only when you must.
Reciprocal Rank Fusion for Hybrid Search
Reciprocal Rank Fusion merges two ranked lists without tuning score weights. The RRF formula, why the k constant matters, and how to run it in OpenSearch.
Metadata Filtering in RAG Vector Search
Post-filtering vector search silently drops good results. Here is pre-filter vs filtered ANN for RAG, why filtered HNSW is hard, and how to choose.
Contextual Retrieval for RAG, Step by Step
Vector search misses chunks that lost their document context. Contextual Retrieval prepends a short LLM-written summary to each chunk before you embed it.
Tracing a RAG Pipeline with OpenTelemetry
Trace a RAG pipeline with OpenTelemetry: instrument FastAPI, put each retrieval and model step in its own span, and read the latency waterfall in OpenSearch.
HyDE: Hypothetical Document Embeddings for RAG
Vector search fails when a short question looks nothing like its answer. HyDE has an LLM draft a fake answer, embeds that, and retrieves against it instead.
Prompt Injection Defense for RAG Systems
Retrieved documents are untrusted input. A practical guide to defending a RAG copilot against direct and indirect prompt injection, and why filters alone fail.
LLM-as-a-Judge: Scoring RAG Answer Quality
Retrieval metrics say the right docs came back, not that the answer is right. Build an LLM-as-a-judge to score RAG answers for faithfulness and quality.
Fan Out Concurrent LLM Calls with asyncio.gather
Awaiting retrieval and LLM calls one by one wastes seconds per request. Here's how to fan them out with asyncio.gather, bound it, and handle partial failures.
Product Quantization for Vector Search
HNSW makes vector search fast, but the embeddings still fill your RAM. Product quantization compresses them ~32x with a small recall hit. Here is how it works.
Build a Semantic Cache for LLM Apps
Semantic caching for LLM apps: cache answers by embedding similarity in FastAPI, tune the cutoff, and avoid false cache hits. Working code and failure modes.
Matryoshka Embeddings: Smaller Vectors for RAG
Matryoshka embeddings front-load meaning into a vector's first dimensions, so you can truncate them for cheaper, faster RAG search without losing much recall.
Late Interaction Retrieval for RAG with ColBERT
A bi-encoder averages token detail away; a cross-encoder is too slow to rank a corpus. Late interaction with ColBERT sits between them. Here is how it works.
RAG Vector Search in Postgres with pgvector
Store and query RAG embeddings inside Postgres with pgvector: HNSW indexing, distance operators, metadata filtering, hybrid search, and the tradeoffs I hit.
Contextual Retrieval for RAG Pipelines
Chunking a document for RAG strips each piece of its context. Contextual retrieval adds an LLM-written note to every chunk before you index it.
Metadata Filtering in Vector Search for RAG
Adding a metadata filter to a vector search can silently return fewer results or wreck recall. How post-filter, pre-filter, and filterable HNSW actually differ.
Managing the Context Window in Long Agent Runs
An LLM agent that runs long enough fills its context window and starts to slow or fail. How to prune, compact, and offload context so agents keep going.
LLM Sampling: Temperature, Top-p, and Top-k
Temperature, top-p, and top-k are the three knobs that shape how an LLM picks each token. How each one works, when to reach for it, and how they interact.
How to Choose an Embedding Model for RAG
The embedding model sets the ceiling on RAG retrieval quality. How to choose one by task fit, sequence length, dimensions, and domain, plus the silent bugs.
Query Rewriting for Better RAG Retrieval
Short, vague, follow-up questions don't match how your docs are written. How query rewriting, multi-query expansion, and HyDE fix retrieval before it runs.
LLM-as-a-Judge: Evaluating LLM Output Quality
Human review does not scale for grading LLM answers. How to use an LLM as a judge: write a rubric, score with structured output, and control the biases.
Indirect Prompt Injection in RAG Systems
A RAG copilot reads tickets and logs, so whoever writes them can plant instructions in the prompt. How indirect prompt injection works and how to contain it.
How HNSW Vector Search Actually Works
Every RAG stack leans on HNSW but treats it as a black box. Here is how the layered graph index finds nearest neighbors fast, and the knobs that matter.
Cross-Encoder Reranking for RAG Pipelines
Retrieval puts the right chunk at rank 8, but the generator only reads the top few. How a cross-encoder reranker reorders RAG candidates, and where it fails.
Chunking Strategies for RAG Pipelines
How to chunk documents for a RAG pipeline: why fixed-size splitting fails, structure-aware splitting, size and overlap tradeoffs, and the failure modes.
Measuring RAG Retrieval Quality: recall@k, MRR
Changed your embeddings or added a reranker? Measure it. Build a golden set and score retrieval with recall@k, MRR, and nDCG before you trust the change.
Hybrid Search for RAG: BM25 + Vectors
Vector search alone misses exact IDs and error codes. Here's how to combine BM25 keyword search with dense retrieval, fuse the rankings with RRF, and rerank.
Incremental Indexing for RAG Pipelines
How to keep a RAG index fresh without full rebuilds: detect changed files with checksums, re-embed only what changed, and handle deletions safely.
CloudCanvasAI
AI-powered document creation platform pairing Claude with a live .docx / .pptx / .pdf / .xlsx preview. Chat on the left, documents render live on the right, each user gets their own isolated E2B sandbox.
Argusa AI Challenge 2025 · Runner-Up 🥈
Secured 2nd place at the Argusa AI Challenge 2025 by building ARGRAG, a RAG system for complex enterprise document corpora with multi-modal analysis, metadata intelligence, and real-time performance.
Archi · LLM Copilot for CMS Ops
Partnering with MIT to build an LLM-powered RAG copilot for CMS computing operations.