2026-09-17, Rank Tracking · SERP API · Google

Competitor SEO tracking: positions, SERP features, ads and AI Overview citations over time

Competitor SEO tracking is rank tracking run for every domain on the page instead of only yours: for a fixed keyword set, record each competitor’s organic positions, the ads they run, the SERP features they hold and the AI Overview citations they earn, then diff the snapshots over time. The unit of storage is one results page per keyword, market, device and run. The headline metric is share of SERP, which counts all of those placements alongside blue links.

What this does not include: traffic, clicks, backlinks, search volume or keyword difficulty. A results page does not contain them and this API does not return them. Everything below is observed placement.

What to collect per results page

One Google Search request (POST /v1/monitor/google, or a GOOGLE async task) returns every placement on the page as its own field under result:

Keep the raw result JSON too. When you add a competitor or fix an alias, you recompute from snapshots instead of re-collecting history, which you cannot do.

Data model

Five tables cover it. Competitors carry every domain, store name and business name they use.

create table competitor_keys (
  competitor text not null,
  kind       text not null check (kind in ('domain', 'store', 'business')),
  value      text not null,              -- 'billfox.com', 'BillFox Store', 'BillFox Austin'
  primary key (kind, value)
);

create table keywords (
  keyword_id bigint primary key,
  text       text not null,
  country    text not null,
  location   text,                        -- 'Austin,Texas,United States' or null
  device     text not null default 'desktop',
  grp        text not null                -- 'category', 'comparison', 'brand', 'problem'
);

create table snapshots (
  task_id    uuid primary key,
  keyword_id bigint not null references keywords,
  run_date   date not null,
  sample     int  not null default 0,
  status     text not null,
  result     jsonb,
  unique (keyword_id, run_date, sample)
);

create table placements (
  task_id    uuid not null references snapshots,
  kind       text not null,               -- 'organic', 'ad_top', 'ad_other', 'shopping_ad', 'aio_cite', 'paa', 'local', 'shopping'
  rank       int  not null,               -- order within its kind
  owner_kind text not null,               -- 'domain', 'store', 'business'
  owner      text not null,
  url        text,
  primary key (task_id, kind, rank)
);

create table slot_weights (
  kind   text not null,
  rank   int  not null,                   -- 0 = any rank
  weight numeric not null,
  primary key (kind, rank)
);

placements is derived from snapshots.result and can be rebuilt at any time. sample lets you take several readings a day for ads without duplicating organic data.

Share of SERP

Organic share of voice weights organic results by a click curve. That misses a competitor who is cited in the AI Overview and owns the top ad while ranking seventh. Share of SERP counts every placement type with a weight you set:

share_of_SERP(D) = Σ weight(placements owned by D) ÷ Σ weight(all placements)

over one keyword set, market, device and period. Two rules make it comparable over time:

1. The denominator includes every owner, tracked or not. A new entrant should shrink everyone’s share instead of being invisible.
2. Weights are fixed for the life of a series. Change them and you start a new series.

A starting weight table, to be replaced with your own judgement or click data:

These weights are illustrative, not measured click-through rates. Use the same weights every run.

-- Share of SERP per owner for one week, sample 0 for organic features, all samples for ads
with p as (
  select pl.*, s.sample
  from placements pl
  join snapshots s using (task_id)
  join keywords k using (keyword_id)
  where s.status = 'COMPLETED'
    and s.run_date between '2026-09-07' and '2026-09-13'
    and k.grp in ('category', 'comparison')
    and (pl.kind in ('ad_top', 'ad_other', 'shopping_ad') or s.sample = 0)
),
w as (
  select coalesce(c.competitor, 'other:' || p.owner) as owner,
         coalesce(sw.weight, sw_any.weight, 0) as weight
  from p
  left join slot_weights sw     on sw.kind = p.kind and sw.rank = p.rank
  left join slot_weights sw_any on sw_any.kind = p.kind and sw_any.rank = 0
  left join competitor_keys c   on c.kind = p.owner_kind and c.value = p.owner
)
select owner, sum(weight) / sum(sum(weight)) over () as share_of_serp
from w group by owner order by share_of_serp desc limit 20;

Report share of SERP beside its components (organic share, ad presence rate, AI Overview citation rate). A rising total driven only by ads means something different from one driven by citations.

Scheduling with async batches and webhooks

Submit each run as async tasks: no synchronous surcharge, up to 500 tasks per POST /v1/async/task/batch, results pushed to your webhook. The idempotency key encodes the slot, so a crashed scheduler can resubmit safely and duplicates come back as RESOURCE_ALREADY_EXISTS.

const API = "https://api.answerline.dev";
const headers = { Authorization: `Bearer ${process.env.API_KEY}`, "Content-Type": "application/json" };

type Keyword = { keyword_id: number; text: string; country: string; location: string | null; device: string };

export async function submitRun(keywords: Keyword[], runDate: string, sample: number, withAio: boolean) {
  const tasks = keywords.map((k) => ({
    taskType: "GOOGLE",
    idempotencyKey: `cseo:${k.keyword_id}:${runDate}:${sample}`,
    webhook: { url: "https://seo.example.com/hooks/serp" },
    payload: {
      query: k.text,
      country: k.country,
      device: k.device,
      ...(k.location ? { location: k.location } : {}),
      ...(withAio ? { include: { aioverview: { markdown: false } } } : {}),
    },
  }));
  const accepted: { taskId: string; keywordId: number }[] = [];
  for (let i = 0; i < tasks.length; i += 500) {
    const chunk = tasks.slice(i, i + 500);
    const res = await fetch(`${API}/v1/async/task/batch`, { method: "POST", headers, body: JSON.stringify(chunk) });
    if (!res.ok) throw new Error(`batch at ${i}: HTTP ${res.status}`);
    const body = await res.json();
    for (const r of body.results) {
      if (r.success) accepted.push({ taskId: r.task.id, keywordId: keywords[i + r.index].keyword_id });
      else if (r.error.code !== "RESOURCE_ALREADY_EXISTS") console.warn(chunk[r.index].idempotencyKey, r.error.code);
    }
  }
  return accepted; // insert into snapshots with status 'QUEUED'
}

A typical schedule:

The webhook body is {task, credits, response}, with the Google result in response.result. Verify the Webhook-Signature header against the raw body, skip deliveries with "test": true, and treat task.id as the deduplication key because deliveries can repeat. Verifying webhook signatures has the code; async and webhooks have the rules.

Turning a result into placements

const host = (u?: string) => {
  try { return new URL(u ?? "").hostname.replace(/^www\./, ""); } catch { return ""; }
};

export function placements(result: any) {
  const rows: { kind: string; rank: number; owner_kind: string; owner: string; url?: string }[] = [];
  const organic = [...(result.organicResults ?? [])].sort(
    (a, b) => (a.page ?? 1) - (b.page ?? 1) || (a.position ?? 0) - (b.position ?? 0));
  organic.forEach((o, i) => rows.push({ kind: "organic", rank: i + 1, owner_kind: "domain", owner: host(o.link), url: o.link }));

  const counters: Record<string, number> = {};
  for (const a of result.ads ?? []) {
    const shopping = a.type === "SHOPPING_CARD";
    const kind = shopping ? "shopping_ad" : a.blockPosition === "top" ? "ad_top" : "ad_other";
    counters[kind] = (counters[kind] ?? 0) + 1;
    rows.push({ kind, rank: counters[kind], owner_kind: shopping ? "store" : "domain",
                owner: shopping ? a.store ?? "" : a.domain ?? host(a.url), url: shopping ? undefined : a.url });
  }

  const seen = new Set<string>();
  for (const c of result.aioverview?.citationPills ?? []) {
    if (seen.has(c.url)) continue;
    seen.add(c.url);
    rows.push({ kind: "aio_cite", rank: seen.size, owner_kind: "domain", owner: c.domain, url: c.url });
  }

  (result.peopleAlsoAsk ?? []).filter((q: any) => q.type === "LINK" && q.link)
    .forEach((q: any, i: number) => rows.push({ kind: "paa", rank: i + 1, owner_kind: "domain", owner: host(q.link), url: q.link }));
  (result.localResults ?? []).forEach((l: any) =>
    rows.push({ kind: "local", rank: l.position, owner_kind: "business", owner: l.title }));
  (result.shoppingCards ?? []).forEach((s: any, i: number) =>
    rows.push({ kind: "shopping", rank: i + 1, owner_kind: "store", owner: s.store ?? "" }));
  return rows;
}

Citation pills can repeat a URL across pills, so the loop counts each cited URL once per page.

Change detection

Diff each completed snapshot against the previous run for the same keyword, market, device and sample index. Events worth storing:

Result pages fluctuate. Two rules keep the feed readable:

1. Persistence. Emit an alert when the new state holds for two consecutive runs; store single-run flips but do not page anyone.
2. Aggregation. A competitor entering the top 10 on 40 category keywords the same week is one alert with 40 keywords, not 40 alerts.

-- Competitors that entered the top 10 on two consecutive runs after being absent
with ranks as (
  select s.keyword_id, s.run_date, c.competitor, min(p.rank) as best
  from snapshots s
  join placements p using (task_id)
  join competitor_keys c on c.kind = 'domain' and c.value = p.owner
  where p.kind = 'organic' and s.sample = 0 and s.status = 'COMPLETED'
  group by 1, 2, 3
),
runs as (
  select distinct keyword_id, run_date,
         lag(run_date, 1) over (partition by keyword_id order by run_date) as prev1,
         lag(run_date, 2) over (partition by keyword_id order by run_date) as prev2
  from snapshots where sample = 0 and status = 'COMPLETED'
)
select r.keyword_id, now_.competitor, r.run_date
from runs r
join ranks now_ on now_.keyword_id = r.keyword_id and now_.run_date = r.run_date and now_.best <= 10
join ranks p1   on p1.keyword_id = r.keyword_id and p1.run_date = r.prev1 and p1.competitor = now_.competitor and p1.best <= 10
left join ranks p2 on p2.keyword_id = r.keyword_id and p2.run_date = r.prev2 and p2.competitor = now_.competitor and p2.best <= 10
where p2.keyword_id is null;

Feature-level changes that are not about a competitor, such as an AI Overview appearing on a keyword at all, are covered in SERP features change tracking.

Cost from credits

Credits per async task: Google Search 3 for one page, plus 2 per extra page, plus 2 once when include.aioverview is set. Synchronous calls add 2. A failed request is charged nothing (credits).

Worked example: a B2B software company tracks six competitors in the US:

Moving the 600 category keywords from daily to weekly cuts the first line to 12,000 credits and the total to 28,400. Decide per series whether you act on a change within a day; if not, weekly is enough. Ads on brand terms are the series where repeated daily samples matter most, because advertisers can differ from one search to the next. Plan allowances are on pricing, and AI monitoring cost planning covers budgeting when AI answers join the same account.

Pitfalls

• Keying shopping ads by URL. Sponsored shopping cards carry a redirect in url; use store.
• Competitors with many domains. Blogs, docs subdomains and country sites belong in competitor_keys, or their share is understated.
• Mixing devices or locations in one series. Mobile and desktop pages differ, and localResults is desktop only.
• Organic counted once per ad sample. Restrict organic and feature placements to sample 0.
• Treating share of SERP as traffic share. It is weighted placement, not clicks. Keep traffic claims out of reports built on it.
• Reporting before the run completes. Reconcile tasks still QUEUED or PROCESSING with GET /v1/async/task/{taskId} before computing the day’s metrics.

The per-domain rank model is detailed on rank tracking, and the metric set for rivals, including AI answer citation share, on competitor analysis. To send a first request, use the quickstart.

Keep reading