Content Signals

Content Signals is an extension to robots.txt that separates permission to crawl from permission to use. A publisher adds a Content-Signal line declaring intent for three distinct purposes: search (including the page in a search index and linking to it), ai-input (using the page as grounding material for a generated answer), and ai-train (using it to train or fine-tune a model). Each is set to yes or no, so a site can welcome search and AI citation while declining training.

The distinction matters because the classic robots.txt vocabulary cannot express it. Disallowing a crawler to prevent training also removes the content from retrieval, which removes it from AI answers. Many publishers spent 2024 and 2025 discovering that a blanket block protected their content and erased their AI search visibility at the same time.

Cloudflare popularized the format in 2025 and began adding default signals for sites on its network, which pushed the syntax onto a large share of the web quickly; see our Cloudflare integration write-up for how that rollout affects crawler behavior. Content Signals is a preference expressed in plain text, not an enforcement mechanism: it carries the weight of a stated license term, and compliance is voluntary in the same way robots.txt compliance is. Publishers wanting stronger footing generally pair it with RSL for formal licensing and pay per crawl or bot rules for enforcement.

For GEO teams, Content Signals is the lowest-effort way to make an accurate declaration: one line of text that says "cite me, don't train on me" instead of an all-or-nothing block that costs citations.