Chunking

A retrieval system does not search documents. It searches pieces of documents, and those pieces were made by a decision you took before any query arrived. A passage that was split down the middle can never be retrieved whole.

The trade is straightforward to state and hard to settle. Small chunks are precise — the retrieved text is almost all relevant — and they cut the answer away from the context that makes it mean something. Large chunks preserve context and bury the answer among paragraphs that dilute the match.

The lab holds three chunkings of the same corpus. Switch between them on one question and watch the scores move.

• By section: one chunk per heading. Respects boundaries a human already chose, which is usually the best default.
• Small: one or two sentences. Higher precision, and a retrieved sentence often no longer says what it is about.
• Whole document: maximum context, lowest precision. On a question with a specific answer, the right document can rank below a wrong one.

One detail is worth more than the size decision: keep the heading with the body. "Furniture, mattresses and items over 25kg have a 14 day return window" is far more retrievable when preceded by "Furniture and large items", because the heading states the topic the body assumes.