> ## Documentation Index
> Fetch the complete documentation index at: https://docs.chonkie.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshooting

> Solutions to common issues when using Chonkie

Having trouble? Check the common issues below. If your problem isn't listed, reach out on [Discord](https://discord.gg/Q6zkP8w6ur).

## Installation Issues

<AccordionGroup>
  <Accordion title="ModuleNotFoundError: No module named 'chonkie.embeddings'">
    You're trying to use a chunker that requires the `embeddings` extra. Install with:

    ```bash theme={"system"}
    pip install "chonkie[all]"
    ```

    Or for just embeddings support:

    ```bash theme={"system"}
    pip install "chonkie[semantic]"
    ```

    See the [installation guide](/oss/installation) for all available extras.
  </Accordion>

  <Accordion title="ImportError: tree-sitter not found">
    The `CodeChunker` requires tree-sitter. Install with:

    ```bash theme={"system"}
    pip install "chonkie[code]"
    ```

    You also need the language grammar for your target language. Tree-sitter grammars are installed automatically for common languages.
  </Accordion>

  <Accordion title="Conflicting dependency versions">
    If you hit version conflicts, try installing in a fresh virtual environment:

    ```bash theme={"system"}
    python -m venv .venv
    source .venv/bin/activate  # or .venv\Scripts\activate on Windows
    pip install "chonkie[all]"
    ```
  </Accordion>
</AccordionGroup>

## Chunking Issues

<AccordionGroup>
  <Accordion title="My chunks are too small">
    Increase `chunk_size`. The value represents the maximum token count per chunk. Common values:

    * **256 tokens** — tight context windows, Q\&A bots
    * **512 tokens** — balanced default for most RAG pipelines
    * **1024+ tokens** — longer documents, summarization tasks

    ```python theme={"system"}
    chunker = RecursiveChunker(chunk_size=1024)
    ```

    Also check `chunk_overlap` — higher overlap means more content is shared between chunks, which can help retrieval but makes individual chunks shorter.
  </Accordion>

  <Accordion title="My chunks are too large">
    If chunks exceed your expected size, check:

    1. Your `chunk_size` setting — it's in **tokens**, not characters
    2. Some chunkers (SentenceChunker, RecursiveChunker) won't split mid-sentence, so a single long sentence can exceed the limit
    3. For strict limits, use `TokenChunker` which guarantees the size constraint
  </Accordion>

  <Accordion title="SemanticChunker is slow">
    SemanticChunker calls an embedding model for every sentence pair. To speed it up:

    1. **Reduce input size** — pre-split very long documents
    2. **Increase similarity\_threshold** — merges fewer comparisons
    3. **Use a faster embedding model** — `all-MiniLM-L6-v2` is a good balance of speed and quality
    4. **Use batch mode** — `chunker.chunk_batch(texts)` is more efficient than looping

    If speed is critical and you can trade some quality, consider `RecursiveChunker` instead.
  </Accordion>

  <Accordion title="CodeChunker produces empty chunks or errors">
    Common causes:

    1. **Wrong language specified** — make sure the `language` parameter matches your file (e.g., `"python"`, `"javascript"`, `"typescript"`)
    2. **Invalid syntax** — tree-sitter can't parse files with syntax errors. Fix the source or use `RecursiveChunker` as a fallback
    3. **Very small files** — if the entire file is smaller than `chunk_size`, you'll get one chunk containing the whole file
  </Accordion>

  <Accordion title="FastChunker cuts in the middle of words">
    This is expected behavior. FastChunker splits on byte boundaries for maximum throughput. It doesn't understand word or sentence boundaries.

    If you need clean boundaries, use `RecursiveChunker` or `SentenceChunker` instead. FastChunker is designed for pipelines where byte-aligned chunks are acceptable (e.g., pre-filtering before a more precise chunker).
  </Accordion>
</AccordionGroup>

## Integration Issues

<AccordionGroup>
  <Accordion title="Embedding dimension mismatch with vector DB">
    Your embedding model's output dimension must match your vector database's configured dimension.

    Common dimensions:

    * `all-MiniLM-L6-v2`: 384
    * `text-embedding-ada-002` (OpenAI): 1536
    * `text-embedding-3-small` (OpenAI): 1536
    * `voyage-2`: 1024

    Check your vector DB collection was created with the correct dimension. For Chroma, it auto-detects. For Qdrant/Pinecone, you specify at collection creation time.
  </Accordion>

  <Accordion title="Handshake connection errors">
    Verify:

    1. Your vector DB is running and accessible at the configured URL
    2. API keys / auth tokens are correct
    3. The collection/index exists (some handshakes auto-create, others don't)
    4. Network/firewall allows the connection

    Test the connection independently before using Chonkie:

    ```python theme={"system"}
    # Example: test Qdrant connection
    from qdrant_client import QdrantClient
    client = QdrantClient(url="http://localhost:6333")
    print(client.get_collections())
    ```
  </Accordion>

  <Accordion title="Memory issues with large documents">
    For very large documents (100+ MB):

    1. **Use streaming** — process documents in sections rather than loading entirely into memory
    2. **Use batch processing** — `chunk_batch()` processes documents sequentially, keeping memory bounded
    3. **Consider FastChunker** — it processes in a single pass with minimal memory overhead
    4. **Pre-split** — divide large files into sections before chunking

    ```python theme={"system"}
    # Process a large file in sections
    with open("large_file.txt") as f:
        while section := f.read(1_000_000):  # 1MB at a time
            chunks = chunker.chunk(section)
            # process chunks...
    ```
  </Accordion>
</AccordionGroup>

## Async / Concurrency Issues

<AccordionGroup>
  <Accordion title="RuntimeError: cannot schedule new futures after interpreter shutdown">
    This happens when using async chunkers without a running event loop. Make sure you're inside an async context:

    ```python theme={"system"}
    import asyncio
    from chonkie import RecursiveChunker

    async def main():
        chunker = RecursiveChunker(chunk_size=512)
        chunks = await chunker.achunk(text)

    asyncio.run(main())
    ```
  </Accordion>

  <Accordion title="Slow performance with asyncio.gather">
    If you're gathering many chunk operations and it's slower than expected, the thread pool may be saturated. The default pool size is limited. For CPU-bound chunkers, parallelism is bounded by CPU cores:

    ```python theme={"system"}
    import asyncio

    # Limit concurrency to avoid overwhelming the thread pool
    semaphore = asyncio.Semaphore(4)

    async def chunk_with_limit(text):
        async with semaphore:
            return await chunker.achunk(text)

    results = await asyncio.gather(*[chunk_with_limit(t) for t in texts])
    ```
  </Accordion>
</AccordionGroup>