Consistent Hashing: Sharding Without the Reshuffle
Add a cache node with plain modulo hashing and almost every key moves. Consistent hashing moves only one arc. How the hash ring and virtual nodes work.
#DevOps(18)
Splitting an LLM Across GPUs: Tensor vs Pipeline
A model too big for one GPU has to be split. How tensor and pipeline parallelism divide an LLM, what each costs, and when to reach for which.
How Much GPU Memory to Serve an LLM
Pick a GPU for an LLM and it 'fits' on paper, then OOMs under load. A quick way to estimate the VRAM a model needs: weights, KV cache, and overhead.
Kubernetes Native Sidecars: Fix Startup Order
Kubernetes native sidecars are init containers that keep running. They start before your app, restart on their own, and get shut down last. Here is how.
OpenSearch as a Vector Store for RAG
Run RAG retrieval inside the OpenSearch cluster you already operate. Set up a knn_vector index, pick faiss vs lucene, and avoid the memory traps.
PodDisruptionBudgets: Survive a Node Drain
A cluster upgrade drains a node and evicts every replica of your service at once. How a PodDisruptionBudget caps voluntary disruptions and keeps you online.
OpenSearch ISM: Automate Index Rollover
Log indexes grow until a data node runs out of disk. How OpenSearch ISM rolls indexes over by size, ages them through hot and warm, then deletes on schedule.
Autoscaling a FastAPI Service with the Kubernetes HPA
The Kubernetes HPA scales pods from a metric, but its defaults thrash and lag under real load. How the control loop works, how to tune it, and where it breaks.
Scheduling GPU Pods on Kubernetes
A GPU node won't run your Pods until a device plugin advertises it. How Kubernetes discovers GPUs, how to share one across Pods, and where scheduling breaks.
Zero-Downtime Reindexing in OpenSearch
Change an OpenSearch mapping without dropping writes or serving stale data. A step-by-step reindex with aliases, the Reindex API, and the failure modes.
Kubernetes Jobs and CronJobs for Batch Work
Run finite and scheduled work on Kubernetes with Jobs and CronJobs: completions, parallelism, backoffLimit, cleanup, and the failure modes that bite.
ArgoCD ApplicationSets: One Template, Many Clusters
ArgoCD ApplicationSets generate one Application per cluster or environment from a single template. How the generators work, a Git example, and where they bite.
Graceful Shutdown for FastAPI on Kubernetes
A rolling deploy sends SIGTERM and kills your FastAPI pod mid-request, dropping live SSE streams. How to catch it, drain connections, and shut down cleanly.
Scale Kubernetes Workers on Queue Depth with KEDA
A CPU-based HPA can't see a backed-up queue, so workers fall behind. How KEDA autoscales Kubernetes workers on queue depth, and where it breaks.
Kubernetes OOMKilled: Requests vs Limits
Your pod died with OOMKilled and exit 137. What Kubernetes memory requests and limits actually do, what triggers the kill, and how to stop it happening.
Kubernetes Probes: Liveness, Readiness, Startup
Kubernetes has three health probes and mixing them up causes outages. How liveness, readiness, and startup probes work, and the failure modes to avoid.
ArgoCD Sync Waves: Ordering a Kubernetes Rollout
ArgoCD applies your whole app at once, so a migration races the pods that need it. How sync waves and hooks order a Kubernetes rollout, and where they stall.
OpenSearch Dashboards for Workflow Monitoring
Build an OpenSearch dashboard that watches a job pipeline: structured logs, an index mapping, the queries behind each panel, and alerts that fire early.