RAG
Chunking
Split docs before embedding. Sensible defaults; override per doc type.
import { chunkText } from '@agentskit/rag'
const chunks = chunkText(longDoc, {
chunkSize: 800,
chunkOverlap: 120,
})chunkText(text, options) returns string[]. Pass a custom split when you want your own boundaries instead of the default sliding window:
const paragraphs = chunkText(longDoc, {
chunkSize: 800,
chunkOverlap: 120,
split: text => text.split(/\n\n+/),
})#Options
| Option | Type | Default |
|---|---|---|
chunkSize | number | required (createRAG default: 512) |
chunkOverlap | number | required (createRAG default: 50) |
split | (text: string) => string[] | omitted — sliding window over whitespace |
Standalone chunkText has no built-in size defaults. When you go through createRAG, unspecified chunkSize / chunkOverlap become 512 / 50.
If split is provided, its return value is used as-is (empty strings dropped). chunkSize and chunkOverlap then apply only to the default splitter.
#Rules of thumb
- Prose: 800 / 120.
- Markdown: 1200 / 150.
- Code: 1500 / 0.