2026-09-18, GEO · Fundamentals · Tutorials

Run your own AI citation study: a defensible methodology

An AI citation study is a repeated, documented measurement of which sources AI engines cite for a defined set of prompts, engines, markets and dates. A number about AI search that you have to defend to a client, a board or a journalist is best measured yourself, with a method others can inspect. Below: design, collection through the API, the statistics, and a checklist for publishing.

Definitions

Write these down before collecting anything, and use them unchanged in the report.

• Answer: one engine response to one prompt in one market at one time; one result object.
• Sourced answer: an answer whose result.sources[] is not empty.
• Trigger rate: sourced answers divided by all answers. For Google AI Overviews, requests with include.aioverview whose result.aioverview is not null, divided by all requests.
• Citation rate (for a domain): sourced answers that cite the domain at least once, divided by sourced answers.
• Citation share (for a domain): source URLs on the domain, divided by all source URLs in the study.
• Mention rate (for a brand): answers whose text names the brand, divided by all answers.
• Source mix: the distribution of all source URLs over domain types you define, such as forum, publisher, review site, vendor, documentation, video.

Rate measures how often a reader can find the domain; share measures how much of the evidence comes from it. A domain can have a high rate and a low share when it appears in many answers as one source among many.

Step 1: state the question and the population

A study answers a question about a population of prompts, not about AI search in general. Keep the question narrow: “In US English, how often do ChatGPT and Gemini cite review sites for B2B accounting software comparison prompts, in September 2026?” That sentence fixes the category, intents, engines, market and window, which is everything a reader needs to judge whether the result applies to them.

Step 2: build a stratified prompt basket

Random prompts measure nothing useful. Build the basket from real demand and stratify it by intent (category, comparison, problem, branded) so each cluster can be reported on its own. Weight clusters by how often buyers ask them if you want one blended number, and say how you weighted. Designing a prompt set covers sourcing the prompts.

Every rate is a proportion, and the worst-case 95% margin of error for a proportion from n independent observations is 1.96 × √(0.25 / n):

A “cell” is one combination you report separately, such as engine × market × cluster. If you want to compare two engines across four intent clusters with ±10-point precision, that is roughly 100 prompts in each of eight cells, not 100 prompts in total. Sampling vs census covers how to size a test that detects a change between two periods.

Step 3: fix engines, markets and settings

• Engines. Choose the ones your audience uses. Each engine has its own retrieval, so do not pool them into one “AI” number.
• Markets. Set country on every request. ChatGPT and Gemini answer from 213 countries, Copilot, Perplexity and Grok from any country, and Gemini also takes a US state (state-level answers). Numbers from one market do not transfer to another.
• Settings. Record every flag. On ChatGPT, the engine decides whether to search and disableWebSearch changes nothing today; report citation metrics over answers with sources (provenance.webSearchTriggered is true) and state the share without. Do not plan a fan-out analysis on ChatGPT: include.searchQueries is free, but the list comes back empty on the answers served today.

Step 4: decide repetitions and the window

Answers to the same prompt vary between runs (why one check is an anecdote). Run each prompt at least three times per cell, spread across days rather than within one hour, and publish the collection window. A study run in one week describes that week; engines change their models and sources over time.

Step 5: estimate the cost

Prices come from the billing docs. Each request costs the base price, at the Starter tier: ChatGPT $0.0006, Gemini $0.0006, Copilot $0.00274, Perplexity $0.0015, Grok $0.00218, AI Mode $0.00206, Google Search $0.00103, Google News $0.0002.

Worked example: 100 prompts × 3 runs × (ChatGPT $0.0006 + Gemini $0.0006) = 100 × 3 × $0.0012 = $0.36 for one market. A Google AI Overview trigger measurement on the same prompts ($0.00103 plus $0.00071 for include.aioverview) is 100 × 3 × $0.00174 = $0.522 more. A task that fails is not charged. Check your balance with GET /v1/credits before submitting, and see cost planning and pricing for the tiers.

Step 6: collect with async batches

Submit the whole basket as tasks, up to 500 per batch, with idempotency keys that encode prompt, engine, market and run. The keys make resubmission after a network error safe, and they label every stored answer with its place in the design.

import os, requests

API = "https://api.answerline.dev"
HEADERS = {"Authorization": f"Bearer {os.environ['ANSWERLINE_API_KEY']}"}
ENGINES = ["CHATGPT", "GEMINI"]

def build_tasks(prompts: list[dict], country: str, run: int, hook: str) -> list[dict]:
    """prompts: [{"id": "cmp-014", "text": "...", "cluster": "comparison"}]"""
    return [
        {
            "taskType": engine,
            "payload": {"prompt": p["text"], "country": country},
            "idempotencyKey": f"study-2026-09:{engine}:{country}:{p['id']}:run{run}",
            "webhook": {"url": hook},
        }
        for p in prompts
        for engine in ENGINES
    ]

def submit(tasks: list[dict]) -> list[dict]:
    rejected = []
    for start in range(0, len(tasks), 500):
        chunk = tasks[start:start + 500]
        r = requests.post(f"{API}/v1/async/task/batch", json=chunk, headers=HEADERS, timeout=120)
        r.raise_for_status()
        for item in r.json()["results"]:
            code = None if item["success"] else item["error"]["code"]
            if code and code != "RESOURCE_ALREADY_EXISTS":  # already exists = submitted earlier
                rejected.append({"key": chunk[item["index"]]["idempotencyKey"], "code": code})
    return rejected

A 429 QUEUE_LIMIT_EXCEEDED rejects a whole batch; back off and resubmit the same chunk. Results arrive by webhook as the GET /v1/async/task/{taskId} body. Store each delivery verbatim, keyed by task.id, because deliveries can repeat and arrive out of order (webhook reliability). A task that finishes FAILED has no answer: resubmit it within the window with a new run label or record it as missing.

Step 7: compute the metrics from stored JSON

Never compute during collection. Store raw responses, then derive every metric with one script you can publish.

from urllib.parse import urlparse

def domain(url: str) -> str:
    h = (urlparse(url).hostname or "").lower()
    return h[4:] if h.startswith("www.") else h

def on_domain(url: str, target: str) -> bool:
    d = domain(url)
    return d == target or d.endswith("." + target)

def observe(result: dict, target_domain: str, brand: str) -> dict:
    urls = [s["url"] for s in result.get("sources", []) if s.get("url")]
    hits = [u for u in urls if on_domain(u, target_domain)]
    return {
        "sourced": bool(urls),
        "cited": bool(hits),
        "source_urls": len(urls),
        "domain_urls": len(hits),
        "mentioned": brand.lower() in (result.get("text") or "").lower(),
    }

Mention detection by substring is a baseline; add aliases and product names, and check a sample by hand. ChatGPT’s result.entities[] ({type, name}) lists the brands and products an answer names, but the other engines do not return entities, so cross-engine mention rates have to use text. The brand mention method goes deeper.

For source mix, keep a hand-maintained table mapping domains to types and publish it. Classification rules are part of the method: two studies with different rules can disagree without either being wrong.

Step 8: put intervals on every number

Answers from repeated runs of the same prompt are correlated, so treating 300 answers from 100 prompts as 300 independent observations overstates precision. Resample prompts, not answers:

import random

def prompt_bootstrap(per_prompt: dict[str, list[bool]], draws: int = 2000) -> tuple[float, float, float]:
    """per_prompt: prompt id -> cited flag for each of its runs. Returns (rate, low, high) at 95%."""
    ids = list(per_prompt)
    def rate(sample):
        flags = [f for pid in sample for f in per_prompt[pid]]
        return sum(flags) / len(flags)
    estimates = sorted(rate(random.choices(ids, k=len(ids))) for _ in range(draws))
    return rate(ids), estimates[int(0.025 * draws)], estimates[int(0.975 * draws)]

Report “cited in 32% of sourced answers (95% interval 25-39%, 100 prompts × 3 runs)” rather than “cited 32% of the time” (illustrative figures).

Step 9: report so others can check it

A citable study publishes:

1. The question, population and collection window.
2. The prompt list, with cluster labels, or a documented reason it cannot be shared.
3. Engines, markets, request settings and number of runs.
4. Metric definitions exactly as computed, including the domain-type table.
5. Trigger rates alongside every citation rate, with counts and intervals.
6. Excluded answers (failed tasks, empty responses) and how many.
7. The analysis script, and raw data where licensing allows.

The executive report template shows how to present the results internally.

Pitfalls

• Unlabelled denominators. “Cited in 40%” of all answers and of sourced answers are different numbers. Say which.
• Blending engines. An average across engines describes no engine anyone uses.
• Changing prompts mid-study. Any edit to wording creates a new prompt; start a new series.
• Treating failures as zeros. Missing answers bias rates downward.
• Survivorship in the basket. Building prompts from queries where you already rank inflates your own citation rate.
• Borrowed baselines. Comparing your number with a third-party percentage from a different prompt set, market or month compares two different populations.

Submit your first batch with the async guide, and see the share-of-voice framework for turning these rates into ongoing metrics.

Keep reading