2026-09-15, Brand Monitoring · ChatGPT

How to measure brand mentions in ChatGPT answers

To measure brand mentions in ChatGPT, run a fixed set of prompts through ChatGPT on a schedule and compute four rates from each answer’s structured fields: how often the answer names you (mention rate), how often it cites a page on your domain (citation rate), what fraction of all cited sources are yours (citation share), and how often the answer names one of your products (shopping presence). Size the prompt set from the standard error of a proportion, and keep it fixed across periods so a change in the rate reflects a change in the answers. For picking a tool rather than a method, see ChatGPT visibility tracker; for the same metrics across every engine, see brand mentions per engine.

Definitions

• Mention rate: answers whose text names your brand, divided by answers.
• Citation rate: answers with at least one sources[] URL on your domain, divided by answers.
• Citation share: sources[] entries on your domain, divided by all sources[] entries, summed over the answers.
• Shopping presence rate: answers where a product name in entities[] (type: "product") matches your brand, divided by answers that named any product.

Rate tells you whether you are in the evidence at all; share tells you how much of the evidence is yours. An answer citing twelve pages, one of them yours, counts fully toward your rate and contributes 1/12 to your share. Mentions vs citations explains why the mention and citation signals diverge.

Step 1: request the signals together

curl -X POST https://api.answerline.dev/v1/monitor/chatgpt \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "What are the best running shoes for flat feet?",
    "country": "US",
    "include": {"shopping": true}
  }'

include.rawResponse and include.shopping are free. At the Starter tier, a request with shopping costs $0.0006.

include.ads is free to add to that request. Its list comes back empty on the ChatGPT answers served today. include.searchQueries is free and not needed for the rates; its list also comes back empty today, so compare sources[] across runs to see why a citation appeared or disappeared.

ChatGPT decides on its own whether to search; disableWebSearch changes nothing today. provenance.webSearchTriggered is true when the answer cites sources.

Step 2: extract per-answer signals

Each signal comes from its own fields:

• Mention: your brand or an alias in result.text. result.entities, an array of {type, name} pairs, lists the brands and products the answer names; the code matches the text, so the method carries over to engines without entities.
• Citation: a URL on your domain in result.sources, each with a position.
• Shopping presence: your brand in a result.entities[] item with type: "product". With include.shopping, result.inlineProducts[] carries the products from ChatGPT’s shopping search, with price, link and merchant offers; result.shoppingCards[] comes back empty today.

import re
from urllib.parse import urlsplit


def on_domain(url: str, domain: str) -> bool:
    host = urlsplit(url).netloc.lower().removeprefix("www.")
    return host == domain or host.endswith("." + domain)


def brand_signals(result: dict, aliases: list[str], domain: str) -> dict:
    pattern = re.compile(r"\b(" + "|".join(re.escape(a) for a in aliases) + r")\b", re.IGNORECASE)
    mentioned = bool(pattern.search(result.get("text") or ""))

    sources = [s for s in result.get("sources", []) if s.get("url")]
    mine = [s for s in sources if on_domain(s["url"], domain)]

    titles = [e.get("name", "") for e in result.get("entities", []) if e.get("type") == "product"]
    titles = [t for t in titles if t]

    return {
        "mentioned": mentioned,
        "sources_total": len(sources),
        "sources_mine": len(mine),
        "first_cited_position": min((s["position"] for s in mine if "position" in s), default=None),
        "shopping_present": bool(titles),
        "shown_in_shopping": any(pattern.search(t) for t in titles),
    }

Pass result from a synchronous response, or response.result from a completed async task. Word boundaries in the pattern stop “Acme” from matching “Acmeology”; host matching stops acme.com from matching notacme.com.

Step 3: turn signals into rates

import math


def rates(signals: list[dict]) -> dict:
    n = len(signals)
    shop_n = sum(s["shopping_present"] for s in signals)
    total_sources = sum(s["sources_total"] for s in signals)

    def with_margin(k: int, m: int) -> tuple[float, float] | None:
        if m == 0:
            return None
        p = k / m
        return p, 1.96 * math.sqrt(p * (1 - p) / m)  # 95% normal-approximation margin

    return {
        "answers": n,
        "mention_rate": with_margin(sum(s["mentioned"] for s in signals), n),
        "citation_rate": with_margin(sum(s["sources_mine"] > 0 for s in signals), n),
        "citation_share": sum(s["sources_mine"] for s in signals) / total_sources if total_sources else None,
        "shopping_presence_rate": with_margin(sum(s["shown_in_shopping"] for s in signals), shop_n),
    }

Citation share has no margin in this function on purpose. It is a ratio of counts summed over answers, and sources within one answer are not independent, so the binomial formula understates its uncertainty. Estimate its interval by bootstrap: resample prompts with replacement, recompute the share, and take the 2.5th and 97.5th percentiles.

Report every rate per market, since the same prompt set answers differently by market. ChatGPT answers from 213 countries at country level and refuses state with a 400 (state-level answers).

How many prompts you need

A mention rate estimates a proportion. For a rate p measured on n independent prompts, the standard error is SE = sqrt(p(1-p)/n), and the 95% margin of error is about 1.96 × SE. Computed values:

Solving for n gives n = 1.96² × p(1-p) / m² for a target margin m. At p = 0.5 (the worst case): ±10 pp needs 97 prompts, ±5 pp needs 385, ±3 pp needs 1,068.

Planning rules that follow from the formula:

1. Precision improves with the square root of n. Halving the margin takes four times the prompts: 100 to 400, not 100 to 200.
2. Rates near 50% need the most prompts. p(1-p) peaks at 0.5, so if you expect a rate near 10% or 90%, the same margin costs fewer prompts.
3. A trend check can run on 100 to 200 prompts: ±9.8 to ±6.9 pp in the worst case, enough to see a large move.
4. A number you defend to a customer needs roughly 400 to 500 prompts or more. At 500 prompts the worst-case margin is ±4.4 pp.
5. Comparing two periods needs more than one period’s margin. For two independent samples of 200 at p = 0.5, the difference has SE sqrt(2) × 0.035 = 0.05, a ±9.8 pp margin. With the same prompts in both periods, the samples are paired; a paired test (McNemar’s) on prompts that switched is the right comparison, and is usually tighter.

Run a pilot of 30 to 50 prompts first to check your alias matching, not to report a rate. Prompt set design covers which prompts to include.

Repeated runs are not extra prompts

ChatGPT answers vary between runs, so running each prompt several times is tempting. Runs of the same prompt are correlated, though: a prompt whose answer names you once is likely to name you again. Correlated runs add less information than new prompts.

The standard correction is the design effect: with n prompts, m runs each, and within-prompt correlation ρ, the effective sample size is n × m / (1 + (m − 1) × ρ). An illustrative case: 100 prompts × 3 runs with ρ = 0.5 gives 300 / 2 = 150 effective samples, not 300. At ρ = 1 (every run identical) it is 100; at ρ = 0 it is 300. Estimate ρ from your own data as the intra-class correlation of the mention flag across runs.

Repeated runs are still worth doing on a subset: they tell you which prompts are unstable (answer volatility). For the headline rate, spend on more prompts first.

Step 4: run it repeatably

Submit the set as async batches (up to 500 tasks per request), with an idempotency key derived from the prompt, market and day so a retried submission creates nothing twice:

import datetime
import hashlib
import os

import requests

API = "https://api.answerline.dev"
HEADERS = {"Authorization": f"Bearer {os.environ['API_KEY']}", "Content-Type": "application/json"}


def key(prompt: str, country: str, day: str) -> str:
    digest = hashlib.sha256(f"{prompt}|{country}".encode()).hexdigest()[:16]
    return f"mentions-{day}-{country}-{digest}"


def submit(prompts: list[str], country: str, webhook_url: str) -> None:
    day = datetime.date.today().isoformat()
    tasks = [
        {
            "taskType": "CHATGPT",
            "idempotencyKey": key(p, country, day),
            "payload": {"prompt": p, "country": country, "include": {"shopping": True}},
            "webhook": {"url": webhook_url},
        }
        for p in prompts
    ]
    for i in range(0, len(tasks), 500):
        resp = requests.post(f"{API}/v1/async/task/batch", json=tasks[i : i + 500], headers=HEADERS, timeout=30)
        resp.raise_for_status()
        for item in resp.json()["results"]:
            if not item["success"] and item["error"]["code"] != "RESOURCE_ALREADY_EXISTS":
                print(item["index"] + i, item["error"]["code"], item["error"]["message"])

The webhook body is the same JSON as GET /v1/async/task/{taskId}; verify its signature (how), then store response.result with the prompt, market and date. Recompute rates from the stored results, so a change to alias matching or definitions never requires re-running the API. The rank-tracking pipeline post covers the storage side.

Cost of a run

At $0.0006 per request (Starter tier):

• 200 prompts, one market: 200 × $0.0006 = $0.12 per run.
• 500 prompts, one market: 500 × $0.0006 = $0.30 per run.
• A new account’s 100 free requests cover 100 of these tasks.
• A weekly 200-prompt run is about $0.48 a month.

Cadence decides the monthly total; see monitoring cadence and cost planning.

Pitfalls

• Ambiguous brand names. A brand that is also a common word inflates mention rate. Match aliases with word boundaries, and for hard cases require a product or category term nearby.
• Treating answers without products as misses. When entities has no product, the answer named none; keep those answers out of the shopping presence denominator, or report an “answers with products” rate next to it.
• Changing the prompt set mid-series. A new set confounds the trend with sampling. Add prompts as a new series and run both until you can compare.
• Counting answers without search. An answer with no sources showed no search; compute citation metrics over answers with sources and report the share without.
• Reporting share without rate, or rate without share. Each can move while the other stands still.
• Counting citationPills as extra sources. Pills point into sources (their position is a 1-based index into it, and ChatGPT’s citationPillId is 0-based); use sources for share.

For field-level detail, see /engines/chatgpt and /engines/chatgpt/entities.

Keep reading