Reciprocal Rank Fusion for Hybrid Search
Reciprocal Rank Fusion merges two ranked lists without tuning score weights. The RRF formula, why the k constant matters, and how to run it in OpenSearch.
#Python(46)
Read more →
How Continuous Batching Speeds Up LLM Serving
A naive LLM server holds finished slots idle until the slowest request in the batch drains. Continuous batching refills them every step. How it works.
Prompt Caching with the Claude API
Prompt caching reuses a stable prompt prefix to cut Claude API cost and latency. How breakpoints, TTLs, and cache reads work, and what quietly breaks a hit.
Circuit Breakers for LLM API Calls
When an LLM provider degrades, retries make it worse. A practical guide to adding a circuit breaker in Python: the three states, tuning, and failure modes.
Prefill vs Decode: Two Phases of LLM Inference
The first token is slow, the rest stream fast. Why LLM inference splits into a compute-bound prefill and a memory-bound decode, and what it costs you.
HyDE: Hypothetical Document Embeddings for RAG
Vector search fails when a short question looks nothing like its answer. HyDE has an LLM draft a fake answer, embeds that, and retrieves against it instead.
LLM-as-a-Judge: Scoring RAG Answer Quality
Retrieval metrics say the right docs came back, not that the answer is right. Build an LLM-as-a-judge to score RAG answers for faithfulness and quality.
Fan Out Concurrent LLM Calls with asyncio.gather
Awaiting retrieval and LLM calls one by one wastes seconds per request. Here's how to fan them out with asyncio.gather, bound it, and handle partial failures.
Build a Semantic Cache for LLM Apps
Semantic caching for LLM apps: cache answers by embedding similarity in FastAPI, tune the cutoff, and avoid false cache hits. Working code and failure modes.
Rate Limiting a FastAPI Service with a Token Bucket
Add per-user rate limiting to a FastAPI backend with the token bucket algorithm: an in-process version, an atomic Redis script, 429s, and the failure modes.
How FlashAttention Speeds Up the Attention Layer
Attention is memory-bound, not compute-bound. Here's how FlashAttention uses tiling and online softmax to skip the N×N matrix and run exact attention faster.
Async Connection Pooling in FastAPI
Your async FastAPI app stalls under load, then times out. Here is how SQLAlchemy pool_size and max_overflow really work, and how to size them for Postgres.
How Byte-Pair Encoding Tokenizes Text for LLMs
Byte-pair encoding turns text into the tokens an LLM bills and reasons over. How BPE merges are learned, why token counts drive cost, and where it breaks.
How FastAPI Dependency Injection Actually Works
A practical guide to FastAPI dependency injection: how Depends resolves a graph, yield setup and teardown, per-request caching, and where it leaks.
Late Interaction Retrieval for RAG with ColBERT
A bi-encoder averages token detail away; a cross-encoder is too slow to rank a corpus. Late interaction with ColBERT sits between them. Here is how it works.
FastAPI BackgroundTasks vs a Task Queue
FastAPI BackgroundTasks run inside your web process and disappear on restart. When that is fine, when you need a real task queue, and how to move over.
Metadata Filtering in Vector Search for RAG
Adding a metadata filter to a vector search can silently return fewer results or wreck recall. How post-filter, pre-filter, and filterable HNSW actually differ.
Graceful Shutdown for FastAPI on Kubernetes
A rolling deploy sends SIGTERM and kills your FastAPI pod mid-request, dropping live SSE streams. How to catch it, drain connections, and shut down cleanly.
Managing the Context Window in Long Agent Runs
An LLM agent that runs long enough fills its context window and starts to slow or fail. How to prune, compact, and offload context so agents keep going.
Query Rewriting for Better RAG Retrieval
Short, vague, follow-up questions don't match how your docs are written. How query rewriting, multi-query expansion, and HyDE fix retrieval before it runs.
LLM-as-a-Judge: Evaluating LLM Output Quality
Human review does not scale for grading LLM answers. How to use an LLM as a judge: write a rubric, score with structured output, and control the biases.
How to Build a Semantic Cache for LLM Apps
Exact-match caching misses paraphrases, so LLM bills stay high. Here is how to build a semantic cache with embeddings, a similarity threshold, and its traps.
Build an MCP Server for Your LLM Agent
MCP standardizes how LLM agents reach your tools and data. A hands-on guide to building an MCP server in Python, picking a transport, and where it breaks.
Getting Reliable JSON Out of an LLM
Getting an LLM to return JSON is easy; getting valid JSON every time is not. How to use JSON Schema, constrained decoding, and validation to make it reliable.
LLM KV Cache: Why GPU Memory Runs Out
A long prompt or a long agent run can hit CUDA out of memory, and the KV cache is usually why. How it grows, the per-token math, and how to shrink it.
Handling LLM API Rate Limits: Retries and Backoff
Your LLM backend returns 429s the moment traffic bursts. How to retry with backoff and jitter, respect Retry-After, and pace fan-out to stay under the limit.
FastAPI Event Loop Blocking: Sync vs Async
One blocking call in a FastAPI route stalls every other request, including live SSE streams. Here is how the event loop breaks, and how to keep it free.
How HNSW Vector Search Actually Works
Every RAG stack leans on HNSW but treats it as a black box. Here is how the layered graph index finds nearest neighbors fast, and the knobs that matter.
Cross-Encoder Reranking for RAG Pipelines
Retrieval puts the right chunk at rank 8, but the generator only reads the top few. How a cross-encoder reranker reorders RAG candidates, and where it fails.
Chunking Strategies for RAG Pipelines
How to chunk documents for a RAG pipeline: why fixed-size splitting fails, structure-aware splitting, size and overlap tradeoffs, and the failure modes.
Running LLM-Generated Code in a Sandbox
An LLM that writes and runs code needs real isolation, not a try/except. How to sandbox AI-generated code with E2B microVMs, and the failure modes.
Measuring RAG Retrieval Quality: recall@k, MRR
Changed your embeddings or added a reranker? Measure it. Build a golden set and score retrieval with recall@k, MRR, and nDCG before you trust the change.
Hybrid Search for RAG: BM25 + Vectors
Vector search alone misses exact IDs and error codes. Here's how to combine BM25 keyword search with dense retrieval, fuse the rankings with RRF, and rerank.
Streaming LLM Responses from FastAPI with SSE
Stream LLM output token by token from a FastAPI backend with Server-Sent Events: working code, the EventSource client, proxy buffering, and failure modes.
How an LLM Agent Tool-Calling Loop Works
A practical look at the agent loop behind LLM tools: how the model asks to call a tool, your code runs it, and the result feeds back until the answer is done.
Incremental Indexing for RAG Pipelines
How to keep a RAG index fresh without full rebuilds: detect changed files with checksums, re-embed only what changed, and handle deletions safely.
CloudCanvasAI
AI-powered document creation platform pairing Claude with a live .docx / .pptx / .pdf / .xlsx preview. Chat on the left, documents render live on the right, each user gets their own isolated E2B sandbox.
Argusa AI Challenge 2025 · Runner-Up 🥈
Secured 2nd place at the Argusa AI Challenge 2025 by building ARGRAG, a RAG system for complex enterprise document corpora with multi-modal analysis, metadata intelligence, and real-time performance.
Gemini Alchemy
A Gemini-powered tool that turns abstract ideas and complex concepts into visually appealing mindmaps. FastAPI backend, Streamlit frontend.
What’s Cooking? Predicting Cuisines from Recipe Ingredients
Semester Project - Information Retrieval and Artificial Intelligence - CS317 & CS401 - FAST NUCES (Karachi)
IR Vector Space Model
Vector Space Model for Information Retrieval- Assignment 2 - CS317 - FAST NUCES (Karachi).
IR Inverted Index
Inverted Index Based Boolean Information Retrieval Model - Assignment 1 - CS317 - FAST NUCES (Karachi).
AlgorithmsVisualized
A Jupyter Notebook with Different Graph Based Traversal and Search Algorithms Visualized.
NU-Forum
NUDESK is built to provide students a platform to seek help from seniors. built using React, Python Django, PostgreSQL.
QRAS
QRAS is QR based Attendance System. built using Flask, and Basic JQuery, Js and Bootstrap.
QRSMS
QRSMS - QR based Student Management System. built using React, Python Django, PostgreSQL.