Skip to main content
Overlap Refinery
curl --request POST \
  --url https://api.chonkie.ai/v1/refine/overlap
{
  "text": "<string>",
  "start_index": 123,
  "end_index": 123,
  "token_count": 123
}
The Overlap Refinery adds overlapping context between adjacent chunks for better continuity.

​
Request

​
Parameters

​
chunks
array
required
Array of chunk objects to add overlap to. Must be in sequential order from the same document.
​
overlap_size
integer
default:"128"
Number of tokens to overlap between consecutive chunks.
​
tokenizer
string
default:"gpt2"
Tokenizer to use for measuring overlap.

​
Response

​
Returns

Array of chunks with added overlapping context.
​
text
string
The chunk text with added overlap.
​
start_index
integer
Updated starting position reflecting overlap.
​
end_index
integer
Updated ending position reflecting overlap.
​
token_count
integer
Updated token count including overlap.

​
Examples

from chonkie.cloud import TokenChunker, OverlapRefinery

chunker = TokenChunker(chunk_size=100, chunk_overlap=0)
chunks = chunker.chunk("Your text here...")

refinery = OverlapRefinery(overlap_size=25, tokenizer="gpt2")
refined_chunks = refinery.refine(chunks)
⌘I