Skip to main content
Using Chonkie takes two simple steps: First, install the package. Next, start chunking!
This page covers Chonkie Open Source. To get started with our API, visit the API Reference.

Installation

Python

pip install chonkie
Want more features? Install everything with pip install "chonkie[all]". See Installation for more options.

JavaScript

Install the core package for local chunking
npm install @chonkiejs/core
Chonkie JS provides local support for token and recursive chunking. Other chunkers and features are available through the API.

CHONK! 🦛✨

# First import the chunker you want from Chonkie
from chonkie import TokenChunker

# Initialize the chunker
chunker = TokenChunker() # defaults to using GPT2 tokenizer

# Here's some text to chunk
text = """Woah! Chonkie, the chunking library is so cool!"""

# Chunk some text
chunks = chunker(text)

# Access chunks
for chunk in chunks:
  print(f"Chunk: {chunk.text}")
  print(f"Tokens: {chunk.token_count}")
I