Definition
A technical SEO audit is a structured review of the infrastructure that decides whether search engines and AI crawlers can fetch, render, index and understand a website's pages. It covers server responses, crawl directives, rendering, site architecture, structured data and performance, and it should end in a prioritized list of fixes rather than a single score.
In 2026 an audit has two sets of visitors to check for. Googlebot and Bingbot still feed classic search, and their indexes also ground Google's AI Overviews and AI Mode and Microsoft Copilot. Next to them, AI providers run their own AI crawlers with separate jobs: training crawlers such as GPTBot and ClaudeBot, search index crawlers such as OAI-SearchBot, Claude-SearchBot and PerplexityBot, and user-triggered fetchers such as ChatGPT-User. The difference matters because blocking each type has a different consequence (see citation crawlers vs training crawlers). An audit that only checks Googlebot misses half the picture.
The core areas to check:
- Crawl access: robots.txt rules per user-agent token, plus CDN and firewall rules, which often return 403 to AI bots no matter what robots.txt says.
- Indexability: status codes, redirect chains, noindex tags, canonical URLs and XML sitemaps that list only canonical, indexable URLs with accurate lastmod dates.
- Rendering: Vercel and MERJ reported in December 2024 that GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot and PerplexityBot fetch JavaScript files but do not execute them, so text that only appears after client-side rendering is invisible to them. Googlebot does render. See JavaScript rendering.
- Performance: Core Web Vitals thresholds are unchanged in 2026 (LCP under 2.5 seconds, INP under 200 ms, CLS under 0.1 at the 75th percentile), and Google's page experience documentation confirms its ranking systems use them.
- Structured data: valid schema markup that matches what is visible on the page.
- AI feature controls: since August 31, 2026 every Search Console property has a Search generative AI control that can exclude a site from AI Overviews and AI Mode. Confirm it is set on purpose.
The evidence comes from several places. Search Console covers page indexing, Core Web Vitals and, since June 2026, generative AI performance reports. Bing Webmaster Tools covers Bing and Copilot. A crawler such as Screaming Frog or Sitebulb maps the site. Server or CDN logs are the only place an AI crawler block actually shows up, so filter your AI crawler logs by bot and status code, and check bot identity against published IP ranges before trusting the counts.
Prioritize by impact. Anything that stops revenue pages from being fetched or indexed comes first, then rendering gaps on pages you want cited, then performance and markup issues. Run a full audit after migrations, redesigns, CMS changes and bot-management changes at the CDN, and run lighter checks weekly or monthly. In Promptwatch, the Site Health audit runs weekly against your sitemap inventory and scores the share of URLs with no error-level issue, and the free AI SEO audit tool gives a quick single-page check.
Examples of Technical SEO Audit
- A B2B SaaS audit finds Cloudflare bot rules returning 403 to OAI-SearchBot and PerplexityBot even though robots.txt allows both. After an allow rule is added, the crawler logs show 200 responses for both bots the next day.
- A React storefront loads product descriptions through a client-side API call. Fetching the page with JavaScript disabled returns an empty shell, which is what most AI crawlers receive, so the team moves product copy into server-rendered HTML.
- A publisher's sitemap lists 48,000 URLs, of which 11,000 redirect or carry noindex. Pruning it to canonical, 200-status URLs with real lastmod dates gives Google and Bing a cleaner freshness signal.
- During a quarterly audit, a team discovers someone switched the Search Console generative AI control to exclude. They revert it and annotate the date in their AI Overviews reporting so the dip is explained.
