![\"Indexing](\"/303ef938c41457c773a557ab9dfd0d76/index-build.svg\")
Chunk size is the setting people tune last and regret first. Make chunks big and retrieval gets vague. A 2,000-token chunk embeds down to one vector that averages five different subtopics, so it matches everything weakly and nothing sharply. Make chunks small and search gets precise, but now the model reads a 200-token fragment that starts mid-sentence and drops the definition it needed two paragraphs up. A single number trades answer quality against retrieval quality, and no value of that number wins both.
\nParent document retrieval stops treating those as the same number: you search over small chunks and answer with big ones. This post is for engineers who already have a working RAG pipeline and see retrieval find the right document but hand the model a passage too narrow to answer from. I walk through the two-index setup, a framework-free implementation, how to size the chunks, and the specific ways it breaks in practice.
\nA chunk in a RAG (retrieval-augmented generation) system does two unrelated jobs:
\nChunking strategies usually try to satisfy both with one split, and that is the tension. Search wants a chunk small and focused enough that its embedding means one thing. Generation wants a chunk large enough to stand on its own, with the surrounding sentences, the table header the row belongs to, and the “it” that refers to something named earlier.
\nThe fix is to stop forcing one chunk to do both. Split each document twice. Big parent chunks are what the model reads. Small child chunks, carved out of each parent, are what you embed and search. When a child matches, you do not return the child; you return the parent it came from.
\n
Only the children go into the vector index. The parents go into a plain key-value store keyed by an id that every child carries as metadata. That store can be a dict in memory, a Postgres table, a Redis hash, or whatever you already run. So you have two stores for one document, and the link between them is that parent_id.
Here is the index side without a framework, so the moving parts are visible. It has two splitters, a docstore, and a vector index where the payload on each vector is the child’s parent_id. Watch the last two lines: only the children are embedded.
from dataclasses import dataclass\n\n@dataclass\nclass Child:\n text: str\n parent_id: str\n\ndef split(text: str, size: int, overlap: int) -> list[str]:\n # token- or character-based; use your tokenizer in production\n step = size - overlap\n return [text[i:i + size] for i in range(0, len(text), step)]\n\nparents: dict[str, str] = {} # parent_id -> full parent text\nchildren: list[Child] = []\n\nfor i, parent in enumerate(split(document, size=1800, overlap=200)):\n pid = f\"{doc_id}:p{i}\"\n parents[pid] = parent\n for chunk in split(parent, size=350, overlap=50):\n children.append(Child(text=chunk, parent_id=pid))\n\n# embed ONLY the children\nvectors = embed([c.text for c in children])\nindex.add(vectors, payloads=children) # your vector store's add()The parent text is never embedded, and that is the point people miss on the first read. The doc store is dumb storage, and the vector index only ever sees the small chunks.
\nIf you are on LangChain, its ParentDocumentRetriever wires this up for you. LlamaIndex ships the same idea as its auto-merging retriever. The mechanics underneath are the same, and knowing them is what lets you debug the framework when it does something surprising.
Query time is where the two indexes come together. You embed the query and search the child index for the top-k children. Then you map each hit back to its parent and dedupe. Several children from the same parent will often match one query, and you do not want to send that parent three times.
\n![\"At](\"/33ef387de6e119591212940a2a1b60e7/query-expand.svg\")
The function below walks the child hits in score order, skips any parent it has already added, and stops once it has max_parents parents:
def retrieve(query: str, k_children: int = 8, max_parents: int = 3) -> list[str]:\n hits = index.search(embed([query])[0], top_k=k_children) # sorted by score\n seen, out = set(), []\n for hit in hits:\n pid = hit.payload.parent_id\n if pid in seen:\n continue\n seen.add(pid)\n out.append(parents[pid]) # fetch the whole parent\n if len(out) >= max_parents:\n break\n return outTwo knobs matter here, and they are not the same knob:
\nk_children controls how wide you search. Pull more children than you need, because several will collapse into the same parent.max_parents controls how much text you actually send. Search over eight children, come away with three parents, and keep the context focused.If you set k_children too close to max_parents, you will occasionally get two parents back when you asked for three. That happens when, for example, the eighth-ranked child shared a parent with the second.
The numbers in the code, 1,800-token parents and 350-token children, are a reasonable starting point, not a law. I reason about the two sizes differently.
\nChild size is bounded by your embedding model. Most retrieval models were trained on short passages, and their quality degrades well before their advertised token limit. Children of 250 to 400 tokens stay inside the range the model actually handles well. This is the same concern behind picking an embedding model in the first place: the child is what gets embedded, so the child is what the model’s limits apply to.
\nParent size is bounded by two things pulling against each other. A parent must be big enough that the passage is self-contained, and small enough that three of them do not blow your context budget or bury the answer.
\nThere is a real reason not to just make parents enormous and send one. Models attend unevenly across a long context and reliably lose information stranded in the middle of it, an effect documented in Liu et al.’s “Lost in the Middle”. Three tight parents beat one sprawling one, and both beat twenty fragments.
\nIf your documents have real structure (Markdown headings, sections, a ## every few paragraphs), split parents on those boundaries instead of a fixed token count. That way a parent is a section rather than an arbitrary window.
The happy path is a hundred lines. The failures all live in the seams between the two stores.
\nThe two stores drift apart. The vector index and the doc store are separate systems that have to agree on parent_id. Re-index the documents, change the id scheme, or partially reload one store and not the other, and you get children whose parent_id points at a parent that no longer exists. The retrieve loop then throws a KeyError mid-request, or worse, silently skips the hit.
The fix is to treat the two writes as one operation. Never publish new children pointing at parents you have not written yet, and when you delete a document, delete it from both stores. This is the same zero-downtime reindexing discipline, spread across two stores instead of one.
\nDedup throws away a relevance signal. A parent that matched on three of its children is probably more relevant than one that matched on one. The simple loop above ignores that: it keeps the first parent it sees and moves on. For a lot of corpora that is fine. When it is not, rank the deduped parents by their best child score, or by how many children hit, before you apply max_parents. The strong signal is sitting in the hit counts, and the naive loop just does not read it.
Metadata filters have to live on the children. In any multi-user system you must filter retrieval by fields such as source, tenant, or date. Those filters run against the vector index, which only holds children, so copy the filterable fields onto every child at index time. A filter that only exists on the parent cannot touch a search that only sees children. This trips up anyone who added metadata filtering after the fact and put the fields in the wrong store.
\nReranking gets awkward. A cross-encoder reranker scores a (query, passage) pair, and like any other model it has an input limit. Rerank the small children, before you expand to parents: they fit comfortably, and the scores reflect the precise text that matched. Reranking full parents means feeding long, multi-topic passages to a model whose input window may not even hold them. The score you get back is muddier for the same reason the big chunk was a bad search unit to begin with.
Parent document retrieval is one point on a spectrum. Two neighbors are worth knowing:
\nAll three share one idea: the unit you search is smaller than the unit you send, and you keep a link between them. Which one fits depends on how structured your documents are and how much the answer depends on surrounding context.
\nI lean on this pattern whenever documents are long and the answer depends on context around the match: technical docs, incident write-ups, anything with tables. On ARGRAG, the multimodal RAG system I built for the Argusa AI Challenge, the corpus had exactly that shape. It held enterprise documents where a matched row means nothing without the header and the section it sits under. Searching on small children kept retrieval sharp against specific phrasing. Answering with parents meant the model saw the row and the frame around it.
\nThe mental shift is small, but it clears up a question that has no good answer otherwise. Stop asking “what is the right chunk size” as if one number has to serve search and generation at once. Size the child for the embedding model, size the parent for the language model, and keep a parent_id between them.
Then measure whether it worked, with a proper way of measuring retrieval quality rather than reading a few answers and nodding. The only way to know your two sizes are right is to watch recall and answer quality move together instead of trading off.
\nDiagrams by M. Hassan Ahmed, released under CC0. No external image was used for this post; the figures are original work by the author.
","frontmatter":{"title":"Parent Document Retrieval: Search Small, Answer Big","date":"2026-09-09T00:00:00.000Z","description":"Small chunks search precisely but read like fragments. Parent document retrieval matches on child chunks, then feeds the whole parent to the model.","thumbnail":{"childImageSharp":{"fluid":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsSAAALEgHS3X78AAACE0lEQVQozy3R2W6jMBQGYK7SLBgIYDYvmMVgwEmhkzYkhaTppKmq6eRmpF5W8/5PMW410qff25Fl+Wh1iZWqQMehez7cbTfV09Adh1blYy9Phx/7h+Z0uBv3a7W538oyD+tC1WOV2gTQiU4BLHI5FqsjEzvRnvjq8O3I109JNRTrJy6P9d2Zif3C4RMQTwC7MZhmwszyctPLdSdRDJjpdrxYkrmF1WTpp3aQLb0E2AjY2HSpBRmwKXCIblMN0jbKege3LulstLYiaXrlwsJzk5gOlu397WaU3Q4ltY8LD5dxmssmpyxTNZodVVHSobSL0hbi2kW15fEpiKYAGQ6Oiw0qRlQ8BtnOT3slyvsouw9IqVtIK3giKoFjQVjFsjLJCkwT20sMVz0VM7GlzSVuzkxemHyNm5+oOLjxvY+5bkXatqPXc/Z+ys8j/3hLP9+RrLNS7op6gwm9PtO/V/z5+7+PV/xxQX9eyK1kMxBqufrmVVPVZSmqbp1fn+O2IX6I/ABREg4P6tL858C/jHzYlmNfHvpClGxmBJqNVpBtzahdog4E6zmUps8DIkyYzg01Dqj5heo3JSwvkXgJ+MkivYv4wvA0KxCQdjDuVNpIKgbMbxbwZuGZdhBnFc5byr+gpAnjKoxFQEVEkhnwtLkZ6UuiGqt/9Rap5cwIp7o3Bb7KBXCNpe945Bt1/VglsLw5gOr0H3xmWlqUGuSYAAAAAElFTkSuQmCC","aspectRatio":1.899441340782123,"src":"/static/73688f50ebb030dd3cc9f68a43db401a/40a76/hero.png","srcSet":"/static/73688f50ebb030dd3cc9f68a43db401a/c972b/hero.png 340w,\n/static/73688f50ebb030dd3cc9f68a43db401a/27625/hero.png 680w,\n/static/73688f50ebb030dd3cc9f68a43db401a/40a76/hero.png 1360w,\n/static/73688f50ebb030dd3cc9f68a43db401a/ed396/hero.png 2000w","sizes":"(max-width: 1360px) 100vw, 1360px"}}}}}},"pageContext":{"slug":"/2026-09-09-parent-document-retrieval-rag/","previous":"blog/2026-09-10-maximal-marginal-relevance-rag/","next":"blog/2026-09-13-agentic-rag-when-to-retrieve/"}},"staticQueryHashes":["32046230"]}