Definition
Content chunking is the splitting of a document into smaller segments, called chunks, so that each can be indexed, retrieved and passed to a language model separately. The term covers two related things: a technical step inside retrieval-augmented generation systems, and a writing practice of structuring pages into self-contained sections in the hope that AI systems retrieve them more easily.
In a retrieval pipeline, chunking happens before anything is searched. Documents are cut into pieces, each piece is converted to an embedding, and at query time the system retrieves the closest chunks rather than whole documents. Chunk size is a trade-off: small chunks match narrow questions precisely but lose context, while large chunks keep context but dilute relevance. OpenAI's file search tool, for example, splits files into 800-token chunks with a 400-token overlap by default, and developers can set chunk sizes from 100 to 4,096 tokens (OpenAI docs). Web answer engines use their own methods and do not publish them. Google describes its AI features as retrieving relevant passages from pages in its search index; see sub-document retrieval and passage ranking for the search side.
The writing practice spread through GEO advice in 2024 and 2025: short sections under descriptive headings, paragraphs that do not depend on earlier text, and fixed word ranges per section. Some of that is ordinary good structure, and content atomization covers the editorial version. But Google has pushed back on chunking as an AI tactic. On the Search Off the Record podcast in January 2026, Danny Sullivan said Google does not want people turning content into bite-sized chunks for LLMs. Google's May 2026 guide for its AI features then listed chunking among things site owners can ignore, saying its systems 'are able to understand the nuance of multiple topics on a page and show the relevant piece to users' (Google Search Central).
What survives the debate is structure that helps readers and machines alike:
- Headings that say what the section answers.
- Sections that name their subject instead of relying on 'as mentioned above'.
- Tables and lists for data that is naturally tabular.
- Main content in the server-rendered HTML, since crawlers such as GPTBot, ClaudeBot and PerplexityBot do not execute JavaScript (see JavaScript rendering).
What does not help is cutting a coherent explanation into fragments, forcing every section to the same length, or publishing a second LLM version of a page.
To judge whether your structure is a problem, look at what AI answers quote. If responses cite your page but paraphrase the wrong section, or cite a competitor for a fact that sits halfway down a long section of yours, a clearer heading or a self-contained first sentence for that section is a reasonable fix. If you are building your own RAG system over documentation or a knowledge base, chunking is a real engineering decision, and chunk size and overlap should be tested against real questions with retrieval evaluation.
Examples of Content Chunking
- A developer documentation team building an internal support bot tests 300, 800 and 1,500 token chunks against 200 real support questions and picks 800 tokens with overlap because that setting answers the most questions correctly.
- A legal publisher's tenant rights guide is cited by Perplexity for eviction notice periods, but the quoted sentence comes from the wrong state's section. The team adds the state name to each section heading and first sentence.
- A health publisher considers splitting every article into 150-word Q&A blocks for AI. After reading Google's 2026 guidance, it keeps full articles and instead replaces vague headings such as Overview and More info with descriptive ones.
- An e-commerce site discovers that its product specs load through client-side JavaScript, so GPTBot and ClaudeBot fetch pages with no specs at all. Moving the specs into server-rendered HTML fixes the gap.
