Skip to main content
HubSpot+Cloudflare

Send Cloudflare logs for HubSpot CMS sites

Promptwatch can capture crawler logs when your HubSpot site proxies traffic through your own Cloudflare zone. This is called Origin-to-Origin (O2O) — traffic hits your Cloudflare zone first, then forwards to HubSpot. Configure DNS first, then connect Cloudflare with Logpush or a Worker below. HubSpot serves sites from its own Cloudflare zone and doesn’t share request logs, so adding your own Cloudflare zone in front is how you get crawler logs for a HubSpot site. The Worker setup works on any Cloudflare plan, including Free.
Crawler logs start flowing once setup is complete. Logs from before the setup can’t be imported.
Cloudflare’s HubSpot guide doesn’t list a plan requirement, but HubSpot’s reverse proxy docs mention Cloudflare Enterprise for this setup. If HubSpot’s Verify Reverse Proxy Connection check fails on a non-Enterprise zone, contact support before continuing.

Step 1: Set Up Proxied DNS for HubSpot

Set up proxied DNS for HubSpot in your Cloudflare zone. Follow Cloudflare’s HubSpot guide and HubSpot’s reverse proxy setup guide:
  1. In HubSpot, go to Settings → Website → Domains & URLs and connect your domain as a reverse proxy: click Connect a domain, choose Primary or Secondary, and switch off Connect with HubSpot’s built-in content delivery network (CDN).
  2. Create a proxied CNAME record pointing your hostname to [HubID].sites-proxy.hscoscdn[##].net (see the table below).
  3. Set Proxy status to Proxied (orange cloud).
  4. Under Rules → Transform Rules, create a Modify Request Header rule for your hostname that sets X-HS-Public-Host to your domain (for example www.example.com) and X-HubSpot-Trust-Forwarded-For to true.
  5. In HubSpot, open the Edit menu next to your domain and select Verify Reverse Proxy Connection.
[HubID] is your HubSpot account ID, shown in the account menu in the top right of HubSpot. The rest of the target depends on your account’s data center: For US East accounts, [##] comes from the last two digits of your Hub ID: 00 for 00–10 and 50–59, 10 for 11–19 and 60–69, 20 for 20–29 and 70–79, 30 for 30–39 and 80–89, and 40 for 40–49 and 90–99. For example, Hub ID 123456 becomes 123456.sites-proxy.hscoscdn00.net.
Your domain’s nameservers must be managed at Cloudflare. Use the sites-proxy target, not the group##.sites.hubspot.net CNAME from HubSpot’s SSL pre-provisioning step. If your Cloudflare zone has a zone hold enabled, release it before adding the CNAME record, otherwise HubSpot can’t verify the hostname.
Your DNS must be set up via CNAME — A records are not supported for forwarding traffic through Cloudflare Workers. If your root domain uses A records pointing to HubSpot, connect the www hostname through Cloudflare and redirect the root domain to it.
If your HubSpot site is served on www, the Worker needs a route for www.example.com/*. Auto Deploy only creates a route for the root domain (example.com/*), so add a second route in Cloudflare under Workers Routes that points www.example.com/* to the same Worker. In the Manual tab, use www.example.com/* as the route in Step 6.

Step 2: Create an API Key

Create an API key in Promptwatch. Go to Settings → API Keys in your Promptwatch dashboard.

Step 3: Create a Logpush Job

Go to your Cloudflare dashboard, select your domain and navigate to Analytics & Logs > Logpush > Create a Logpush job.

Step 4: Configure HTTP Destination

For destination select HTTP destination. Enter the following HTTP endpoint URL:
Replace YOUR_API_KEY with your actual API key.

Step 5: Configure Logpush Job

For dataset select HTTP Requests and configure the logpush job as follows:Name: promptwatch-crawler-logsIf logs match:
  • Field: ClientRequestHost
  • Operator: Equals
  • Value: YOUR_DOMAIN (example: app.promptwatch.com)
Send the following fields:Request
  • ClientIP
  • ClientRequestHost
  • ClientRequestMethod
  • ClientRequestReferer
  • ClientRequestURI
  • ClientRequestUserAgent
General
  • WorkerSubrequest (allows us to filter out requests initiated from a Worker, since they will be seen as duplicate)
Performance
  • EdgeStartTimestamp
Response
  • EdgeResponseContentType
  • EdgeResponseStatus
Advanced options: Keep the default settings as is. The timestamp format should be rfc3339, which is the default format.

Optional: Filter on known AI Crawler user agents

Promptwatch identifies AI crawlers automatically and only stores AI crawler visits, all other traffic is discarded. You can safely send all your server logs without worrying about non-crawler data being retained.If you prefer to only forward AI crawler traffic, you can use the user agents listed below to pre-filter on your side. Keep in mind that you’ll need to maintain this list yourself as new crawlers emerge.

Step 6: Verify Cloudflare Proxy is Enabled

Verify your domain’s DNS records have the Cloudflare proxy enabled (orange cloud icon). Logpush for HTTP Requests only captures traffic that flows through the Cloudflare network. If the proxy is off (gray cloud / “DNS only”), Cloudflare never sees the requests and can’t push logs.Go to your Cloudflare dashboard → DNS → Records and make sure the Proxy status shows the orange cloud (“Proxied”) for your domain’s A/AAAA/CNAME records.Cloudflare DNS proxy enabled

Step 7: Finish

Once submitted, your Cloudflare Logpush job will automatically send crawler logs to Promptwatch when an AI crawler visits your website. Depending on how active your website is, this can take a few minutes to appear.
For a full list of API response codes, see API Response Codes.