Chunking Strategies for RAG: A Practical Guide
How to split documents for retrieval-augmented generation: chunk sizes, overlap, and structure-aware splitting that improve answer quality.
Clean before you chunk
Chunking messy text bakes the mess into every embedding. Convert sources with the Batch Document Extractor or PDF to Markdown first, so headers, footers, and page numbers do not pollute retrieval.
Structure-aware splitting
Markdown headings are natural boundaries. Split by section first, then by paragraph if a section is too long. Prepend the section heading to each chunk so it carries its own context.
- Respect headings and lists
- Keep tables intact in one chunk
- Prefix chunks with document and section title
- Use small overlap for long paragraphs
Test with real questions
Write twenty questions users actually ask and check whether the right chunk appears in the top results. Use the Token Counter to keep chunk sizes consistent while you experiment.
Try it free in your browser
- PDF to Markdown — Headings, lists, and structure preserved. Perfect for Obsidian, GitHub, RAG, and LLM context.
- Token Counter — Live token, character, and context-window meter for any prompt. Uses official tiktoken encoders.
- Batch Document Extractor — Drop PDFs, images, Word docs, and text files together. Get clean text out — combined or zipped one-per-file. Browser-side OCR included.
Tools mentioned
Frequently asked
What chunk size should I use?
A few hundred tokens is a common starting point. Test retrieval on real questions and adjust.
Is overlap necessary?
Small overlap helps when ideas span boundaries, but too much inflates storage and duplicates results.
Keep reading
How to Prepare PDFs for ChatGPT, Claude, and Gemini
A practical guide to extracting clean, AI-ready text from PDFs — born-digital and scanned — so ChatGPT, Claude, and Gemini answer accurately and don't waste tokens on headers, footers, and page numbers.
The Best Way to Feed Long Documents to Claude (and Other Long-Context Models)
Claude's 200K-token context is generous, but you'll still want to clean, compress, and structure long documents before sending them. Here's a step-by-step playbook.
Cleaning Whisper Transcripts for AI Summaries
OpenAI Whisper, Otter, and YouTube transcripts are full of timestamps, filler words, and speaker noise. Here's how to strip them before sending to ChatGPT or Claude — and why it matters.