Refines chunks by adding overlapping chunks to the original chunk.

OverlapRefinery

Chonkie

The no-nonsense RAG chunking library. Lightweight, fast, and ready to CHONK your texts

Installing Chonkie and its various components

Overview of the different chunkers available in Chonkie

Split text into fixed-size token chunks with configurable overlap

Split text into chunks while maintaining word boundaries

Split text into chunks while preserving sentence boundaries

Recursively chunk documents into smaller chunks.

Split text into chunks based on semantic similarity

Split text using Semantic Double-Pass Merging for improved context preservation

Split text into chunks based on a late-bound token count

Overview of the different embeddings available in Chonkie

Automatically select the best embeddings handler for your use case

Embed text using SentenceTransformer embedding models

Overview of the different refinery available in Chonkie