What's new
Analysis14 Nov 20265 min read

Overlapping document chunks has a bill you can count

In this example, 200 tokens of overlap on a 10,000-token document split into 1,000-token chunks stores 12,400 tokens, which is 24% more text to embed, and whether that helps is a separate test.

Overlap copies text from the end of one chunk into the start of the next. That can keep material around a boundary intact, but it also changes how much text you process and what a search can return, so the setting should be recorded alongside the embedding model.

Anthropic's retrieval article, read on 14 November 2026, describes splitting documents before embedding and retrieving them. It does not say what overlap suits your files. We use a simple finite document to show what the setting does to the token count.

One finite document shows what overlap costs

Take 10,000 tokens and a maximum chunk size of 1,000. With no overlap, ten chunks cover the document. With 200 tokens of overlap, each new full chunk advances by 800 tokens, and the last chunk closes the remaining gap and can be shorter than the maximum.

Calculated greedy chunks for one assumed 10,000-token document. Final chunks may be shorter. No embedding price or retrieval result is claimed.
Overlap tokensChunksStored text tokensExtra text
010100000%
200131240024%
500191900090%

The 200-token setting gives thirteen chunks and 12,400 stored text tokens. At 500 tokens of overlap it gives nineteen chunks and 19,000 tokens. Those are exact for the stated slicing rule, and they leave out titles, contextual summaries, vectors and index overhead.

The final chunk makes the count slightly awkward

For this rule, the chunk count is one plus the ceiling of the remaining document length divided by the step size, where the step is chunk size minus overlap. Add the overlap once for each boundary to get the total stored text. An overlap equal to the chunk size would make the step zero, so the overlap has to be smaller.

A long-document approximation is fine for an early estimate, but a short document needs the finite count. A file that already fits in one chunk gets no overlap at all, so a fixed percentage uplift applied to every file would overstate that part of a collection.

For a real collection, sum the counts file by file and keep the chunker's version and tokenisation method. Character-based and token-based splitting are different assumptions, and a character estimate should not be labelled a measured token bill.

Retrieved neighbours can repeat the same passage

When neighbouring chunks are retrieved together, their shared passage may appear more than once. Look at a few assembled prompts and count the material after any deduplication. Indexed text and text sent for generation are separate totals, and either can move when overlap changes.

Examples where the answer crosses a chunk boundary are worth keeping, to see whether the required material survives each setting. So are examples whose evidence sits comfortably inside one chunk, otherwise the test is made up of cases chosen to favour overlap.

Surrounding text added after retrieval should be recorded too. A small indexed chunk with a larger retrieved neighbourhood is a different design from embedding a large overlapping chunk. You can compare them, provided the configuration makes the difference visible.

Extra tokens only count against a result

Run each candidate setting on the same question set and source collection, and record evidence coverage, accepted answers, indexed tokens and final prompt tokens. More indexed text is certain under this slicing rule. More useful answers have to be observed.

Re-embedding and ongoing requests belong in separate budgets, since a one-off rebuild and repeated prompt duplication are different costs. If the collection changes often, estimate the part that must be regenerated under your update process rather than assuming every rebuild is complete.

Overlap is easier to justify when your results log names the boundary failures it fixed and the duplicate text it added.