TL;DR
- Scraping the ChatGPT interface gives a more accurate picture of AI visibility than the API, because it records the answer, sources, and page elements a real user sees.
- In Promptwatch's test, the interface and the API (web search off) shared 25.6% of their sources, and 84% of interface answers had sources against 26% of API answers.
- Promptwatch collects every answer from the live interfaces of 11 AI platforms, so its citations, query fan-outs, ads, and shopping results match what your buyers see.
"Why scrape the ChatGPT interface when there's an API?"
It's the question we field most often, usually from someone weighing Promptwatch against a tool that does the opposite. So we tested it. In August 2026, we put the same commercial prompts through both routes on the same day: once through ChatGPT's interface, which is what a real person sees, and once through OpenAI's API. Same provider, same questions, same locale.
They came back describing two different markets. This article walks through what we found, what changes when the API has web search turned on, and the honest case for API tracking.
ChatGPT UI vs API: the main differences
The two methods overlapped on just 25.6% of their sources. Seventy percent of what the interface found never appeared in the API results at all.
| ChatGPT UI | ChatGPT API | |
|---|---|---|
| What you're measuring | What your customers actually see | What the model can generate |
| Prompts that returned sources | 84% | 26% |
| Total citations | 392 | 170 |
| Unique domains cited | 149 | 67 |
| Editorial pages found (listicles, comparisons, reviews) | 48 | 3 |
| Community sources found (Reddit, forums) | 6 | 0 |
| Query fan-out visible | Yes | No search ran, so no sub-queries existed |
| Rich results (images, shopping, local, ads) | Visible | Absent from a text response |
How we ran the test
The test used one category of commercial prompts: platforms that help expats buy property in Europe. Every prompt ran twice on the same day and in the same locale, once in the live ChatGPT interface and once through OpenAI's API.
The API calls ran with web search off, the cheapest way to run prompts through the API at scale. That configuration matters for reading the results, so the section on web-search APIs below covers what changes when retrieval is switched on.
Why the data source matters for visibility tracking
When you monitor brand mentions and citations in AI search, manually or with a tool, the data source decides what you can see. Many monitoring vendors rely on the OpenAI API or similar server-side integrations. Promptwatch collects data from the live ChatGPT interface, the same one your customers use when they ask a question.
That choice compounds over time. Interface data measures what is happening to real users. API data measures what the model can produce in a developer environment, which drifts further from the user experience with every product change OpenAI ships to the interface first. Teams that use AI prompt tracking to decide what to write next need the first kind.
What the API got wrong
It invented a company
We asked: "platform that explains local real estate rules for expats buying homes in Europe."
The API recommended HomeRule Europe, complete with country guides, cost calculators, a vetted-expert directory, and even a revenue model. Confident, detailed, well structured. HomeRule Europe does not exist.
That's the risk of API monitoring: a hallucination becomes a data point. You'd log it in your brand tracker with a high visibility score at position 1, and a fictional competitor would land at the top of your competitive set with no warning.
The interface, same prompt, same minute, returned four real platforms with working links. Two were emerging competitors the client hadn't been tracking. Interface tracking filters this kind of hallucination out on its own, because every cited brand comes with a page the engine actually retrieved.
Wrong answers about real brands are the other side of the same problem, and they only become fixable once you can see where they come from.
“We kept getting a wrong answer about Shutterstock. So I created a monitor with only the prompts that were wrong, to see where the misinformation came from. Then we worked with our agency to reach out to those editors and correct the information.” - Emma Gammons, Director of Global External Comms at Shutterstock
Three in four API answers had no sources
Three out of four API answers arrived with no sources at all: no links, nothing to click, nothing to verify.
If you tracked this category through an API, your report would show almost no citations, and you could reasonably conclude there was little AI visibility worth competing for. Run the same prompts through the interface and 84% come back with sources. The visibility exists. The API configuration simply hides it.
Its "sources" came from training data
With web search off, no retrieval happened. The API still produced 170 citations, because the model typed URLs from memory:
- Idealista: "Especially strong in Spain..."
- Kyero: "Focused on overseas homes..."
87% of those citations point at a homepage. Sixty are a bare domain and nothing else, eleven add only /en, and just eleven point to an actual page.
That's what memory-based citation looks like. The model remembers Idealista exists, but it doesn't know which page answers the question, because it never looked. It generates plausible URLs and retrieves none of them.
What about the API with web search turned on?
This is the strongest counter-argument, and it deserves a direct answer. OpenAI's API can run web search, and a tracker that enables it will see live sources instead of memorized homepages. Two 2026 studies published by SEO software vendors measured how close that gets to the interface.
Study one (data collected August 4, 2026). Roughly 1,000 prompts ran through five AI products, each scraped from the consumer interface and called through the closest available API. For ChatGPT, the API triggered search on 83% of prompts against 88% in the interface, yet it cited 3.1 sources per answer against 12.1. Domain overlap between the two methods was 4.8%, the lowest of the five products tested (full study).
Study two (published April 25, 2026). 35 bottom-of-funnel prompts ran for 10 days in three setups: the ChatGPT interface, the API without search, and the API with search. The interface surfaced 845 unique brands, the API without search 135, and the API with search 580. Brand shares still diverged: one brand held 20% share of voice in the search-enabled API and 5% in the interface (full study).
Both studies come from vendors that sell interface-based tracking, so read their conclusions with that in mind. Their numbers point the same way as ours: web search moves the API closer to the interface, and the citations, sources, and brand order still differ enough to change the decisions you'd make from the data.
The reasons are structural:
- Different models. The API exposes the models OpenAI sells to developers. The interface runs whatever model and system instructions OpenAI ships to users, and those change without an API announcement.
- Different retrieval. In the API, the model decides whether and how often to call the search tool. The interface wraps search in its own source selection and ranking, which the API doesn't reproduce.
- No page elements. An API returns text and a list of sources. Shopping cards, ads, image carousels, and local panels never appear in it.
What only the interface shows
Query fan-out
When someone asks ChatGPT a question, it doesn't run one search. It breaks the question into several sub-queries and searches each one separately, such as "expat home search europe multilingual property portals" and "affordable second homes europe property portals."
Those sub-queries are the demand signal behind the prompt. They show which angles of a topic the engine needs answered, which makes them the richest content signal in AI search. A model with search disabled generates no fan-outs at all.
Fan-out behavior also changes without notice. On August 8, 2026, ChatGPT Search started using the site: operator at scale, jumping from about 0.4% to about 17% of all fan-out queries overnight, while searches per response nearly doubled (ChatGPT site: operator data). A tracker reading the interface caught that shift the day it happened.
Rich results, ads, and shopping
Image carousels, shopping modules, local business panels, and ads only exist in the interface. If your brand or a competitor shows up inside one of them, API monitoring will never surface it.
Ads make the point clearest. ChatGPT now shows sponsored placements inside a share of citation-enabled search responses, and Promptwatch tracks that share daily (ChatGPT ads over time). It's also why any ChatGPT ads tracker has to capture the real interface. The same applies to product cards: to track whether your products appear in ChatGPT shopping recommendations, you need the page the shopper sees.
Crawled, cited, clicked: three signals that need real citations
When Promptwatch scrapes the live ChatGPT interface, every citation it logs is a link ChatGPT actually put in front of a user. You can open it, read the page, and see exactly what content the answer was built on.
Three signals get muddled here, and they measure different things:
- Crawled. Your server and CDN logs show when OpenAI's crawler fetched a page.
- Cited. Citation tracking shows when that page appeared as a source in an answer.
- Clicked. Referral traffic tagged utm_source=chatgpt.com shows when a reader clicked through to you.
A page can be crawled and never cited, or cited and never clicked, and each gap points to a different fix. API monitoring with search off produces none of the three, and with search on it produces citations that may never reach a real user. Joined with AI crawler logs, interface citations show the full path from a bot fetching your page to a person reading it.
“In Promptwatch we are able to see how active the ChatGPT bot is on our client websites and exactly how often our content is being used in responses of LLMs.”
Marijn ten Bulte, Head of Organic Channels at Advise
If you optimize against API citations, you optimize against training data or a developer sandbox. If you optimize against interface citations, you optimize against the pages that actually get read.
The competitors you don't know about yet
API monitoring tends to show the incumbents you could have named yourself. Interface monitoring shows the emerging competitors, the Reddit threads gaining traction, and the listicles outranking you on a query you thought you owned.
There is a structural reason. A model's weights are frozen at its training cutoff, so a competitor that launched after that date doesn't exist inside them. Retrieval is the only route a new entrant has into an answer. In our test, the interface named two emerging platforms the client had never tracked, while the API named one that had never existed.
Tracking real prompts in the interface shows the competitive landscape as it stands this week. You catch new entrants early, see which formats (listicles, Q&A pages, Reddit compilations) are getting cited, and see which content angles drive visibility so you can invest in them.
How Promptwatch collects data
Promptwatch tracks 11 AI platforms, including ChatGPT, Google AI Overviews, Google AI Mode, Gemini, Perplexity, and Claude, by scraping their live user interfaces the way a customer would use them. Monitors run on a daily schedule. When a monitor targets a country, US state, or city, collection routes through a residential proxy in that location with the browser locale and timezone matched, so the engine returns the localized answer a local user would get.
That approach gives you four things API tracking can't:
- Real citations. The actual pages each engine served as clickable links, which you can cross-reference with your crawler logs to see the path from a bot fetching a page to that page being cited.
- Query fan-outs. The sub-queries each prompt breaks into, for example visa requirements, affordable markets, expat-friendly portals, and local rules. Each sub-query is a content opportunity your site may be missing.
- Competitive positions. Where each competitor lands in each answer, position by position, as the landscape shifts from week to week.
- Emerging tactics. New sites and formats (Reddit threads, listicles, user-generated content) that rank high enough to be cited, surfaced as soon as they appear.
Visibility is then measured with citation rate (the share of tracked prompts where you're cited), share of voice (how often you appear against competitors on the same prompts), and a visibility score you can trend over time.
UI vs API tracking FAQ
Is scraping the ChatGPT interface more accurate than using the API?
Yes, for measuring what users see. In Promptwatch's August 2026 test, the interface and the API without web search shared 25.6% of their sources. Two 2026 studies that enabled web search in the API still found different citation counts, sources, and brand rankings.
Does the ChatGPT API with web search return the same results as the interface?
No. In an August 2026 study of about 1,000 prompts, the ChatGPT API with search cited 3.1 sources per answer against 12.1 in the interface, with 4.8% domain overlap. The API also returns no ads, shopping cards, or other page elements.
Why do AI visibility tools show different numbers for the same brand?
Tools collect data differently (API or interface) and score it differently. Prospects regularly report gaps of 5 to 15% between tools. Promptwatch publishes its method: each response is scored on position, context, and competitor presence, and unmentioned responses count as zero.
Can API-based tracking see ChatGPT ads or shopping results?
No. Ads, shopping cards, image carousels, and local panels exist only in the interface, so an API response can't contain them.
Does Promptwatch use the ChatGPT API or the interface?
Promptwatch collects data by scraping the UI interfaces of the LLMs used every day by hundreds of millions of people, including ChatGPT, Gemini, AI Overviews, and Perplexity.
