> ## Documentation Index
> Fetch the complete documentation index at: https://promptwatch.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Getting Started

> What crawler logs are and how they enable the Promptwatch platform

## What Are Crawler Logs?

Crawler logs are server-side HTTP request logs that record when AI crawlers, such as GPTBot, ClaudeBot, PerplexityBot, and Google's AI crawlers visit your website. They're typically sourced directly from your CDN or hosting provider.

Promptwatch correlates these logs with your AI visibility and citation data to go beyond simple tracking. You can analyze crawl trends in real time and over time, understand which pages are being crawled most, and connect crawler activity to actual brand mentions and citation patterns across AI platforms.

## Setup guide

### Step 1: Create an API Key

Create an API key in Promptwatch to authenticate crawler log requests. Each project requires its own API key.

1. Go to [Settings → API Keys](https://app.promptwatch.com) in your project dashboard
2. Click **Create API Key**
3. Copy and securely store your API key

<Warning>
  Store your API key securely. Never commit it to version control or share it publicly.
</Warning>

### Step 2: Choose Your Integration

Select your CDN provider or platform below for step-by-step setup instructions.

#### CDN Providers

<CardGroup cols={2}>
  <Card title="Akamai" icon="https://www.promptwatch.com/docs/integration-logos/card/akamai.webp" href="/docs/crawler-logs/akamai">
    Stream logs from Akamai Edge
  </Card>

  <Card title="AWS CloudFront" icon="https://www.promptwatch.com/docs/integration-logos/card/aws-cloudfront.svg" href="/docs/crawler-logs/cloudfront">
    Stream logs from CloudFront via Firehose
  </Card>

  <Card title="Fastly" icon="https://www.promptwatch.com/docs/integration-logos/card/fastly.jpg" href="/docs/crawler-logs/fastly">
    Stream logs from Fastly CDN
  </Card>

  <Card title="Google Cloud CDN" icon="https://www.promptwatch.com/docs/integration-logos/card/gcp.svg" href="/docs/crawler-logs/gcp">
    Stream logs from GCP via Pub/Sub
  </Card>

  <Card title="Netlify" icon="https://www.promptwatch.com/docs/integration-logos/card/netlify.svg" href="/docs/crawler-logs/netlify">
    Stream logs from Netlify Log Drains (Enterprise)
  </Card>

  <span className="dark:hidden">
    <Card title="Vercel" icon="https://www.promptwatch.com/docs/integration-logos/card/vercel-icon.svg" href="/docs/crawler-logs/vercel">
      Stream logs from Vercel Log Drains
    </Card>
  </span>

  <span className="hidden dark:block">
    <Card title="Vercel" icon="https://www.promptwatch.com/docs/integration-logos/card/vercel-icon-dark.svg" href="/docs/crawler-logs/vercel">
      Stream logs from Vercel Log Drains
    </Card>
  </span>

  <Card title="Cloudflare (Logpush)" icon="https://www.promptwatch.com/docs/integration-logos/card/cloudflare.png" href="/docs/crawler-logs/cloudflare-logpush">
    Push logs from Cloudflare (Enterprise)
  </Card>

  <Card title="Cloudflare (Workers)" icon="https://www.promptwatch.com/docs/integration-logos/card/cloudflare.png" href="/docs/crawler-logs/cloudflare-workers">
    Deploy a Worker to analyze your traffic
  </Card>

  <Card title="Manual Setup" icon="code" href="/docs/crawler-logs/manual">
    Send HTTP request logs manually
  </Card>
</CardGroup>

#### Platform

<CardGroup cols={2}>
  <Card
    title="WP Engine + Cloudflare"
    icon={
  <div style={{ display: "flex", alignItems: "center", gap: 4, marginTop: -8, marginBottom: 10, height: 24 }}>
    <img src="https://www.promptwatch.com/docs/integration-logos/card/wp-engine.svg" width={24} height={24} alt="" style={{ objectFit: "contain" }} />
    <span style={{ fontSize: 12, color: "#6b7280", lineHeight: 1 }}>+</span>
    <img src="https://www.promptwatch.com/docs/integration-logos/card/cloudflare.png" width={24} height={24} alt="" style={{ objectFit: "contain" }} />
  </div>
}
    href="/crawler-logs/wp-engine-cloudflare"
  >
    Deploy a Cloudflare Worker for WP Engine sites
  </Card>

  <Card
    title="Webflow + Cloudflare"
    icon={
  <div style={{ display: "flex", alignItems: "center", gap: 4, marginTop: -8, marginBottom: 10, height: 24 }}>
    <img src="https://www.promptwatch.com/docs/integration-logos/card/webflow.svg" width={24} height={24} alt="" style={{ objectFit: "contain" }} />
    <span style={{ fontSize: 12, color: "#6b7280", lineHeight: 1 }}>+</span>
    <img src="https://www.promptwatch.com/docs/integration-logos/card/cloudflare.png" width={24} height={24} alt="" style={{ objectFit: "contain" }} />
  </div>
}
    href="/crawler-logs/webflow-cloudflare"
  >
    Deploy a Cloudflare Worker for Webflow sites
  </Card>

  <Card
    title="Shopify + Cloudflare"
    icon={
  <div style={{ display: "flex", alignItems: "center", gap: 4, marginTop: -8, marginBottom: 10, height: 24 }}>
    <img src="https://www.promptwatch.com/docs/integration-logos/card/shopify.svg" width={24} height={24} alt="" style={{ objectFit: "contain" }} />
    <span style={{ fontSize: 12, color: "#6b7280", lineHeight: 1 }}>+</span>
    <img src="https://www.promptwatch.com/docs/integration-logos/card/cloudflare.png" width={24} height={24} alt="" style={{ objectFit: "contain" }} />
  </div>
}
    href="/crawler-logs/shopify-cloudflare"
  >
    Deploy a Cloudflare Worker for Shopify stores
  </Card>
</CardGroup>

<Note>
  For a full list of API response codes, see [API Response Codes](/docs/crawler-logs/api-responses).
</Note>