2026-09-17, Keyword Research · Google · SERP API

Keyword research API: SERP data versus search volume data

A keyword research API can mean two different data sets. Volume APIs tell you how often a keyword is searched, from ad-platform data or a vendor’s clickstream model. SERP APIs tell you what the results page looks like for that keyword: the questions Google suggests, the related searches, whether an AI Overview answers it and whom it cites, which features push organic results down, and who ranks. This API is the second kind and returns no search volume; the next section covers where to get volume.

The two are complements. Volume tells you how big a keyword is. The SERP tells you what winning it takes and whether there are clicks left to win.

Definitions

• Search volume: an estimate of how many times a keyword is searched per month in a market.
• SERP feature: any block on the results page other than a plain organic result, such as People Also Ask, a local pack, shopping results, a knowledge panel or an AI Overview.
• Search intent: what the searcher wants to do (learn, compare, buy, go somewhere), inferred from the query and from what the results page chooses to show.
• Keyword expansion: growing a seed list into related keywords, from suggestions, related searches, questions or AI query fan-out.

Where volume comes from (and what this API does not do)

This API does not return search volume, keyword difficulty scores, CPC estimates or trend lines. Nothing in its response is a count of searches.

The first-party source for volume is Google Ads. The Google Ads API’s KeywordPlanIdeaService.GenerateKeywordIdeas method generates keyword ideas with historical metrics, including average monthly searches and competition level (Google Ads API docs, checked 2026-09-17). It requires Google Ads API credentials, including a developer token. Third-party SEO tools publish their own volume estimates built from their own data; they are models, and they disagree with each other.

Use a volume source for sizing and prioritisation. Use SERP data for everything below.

What the SERP tells you

One POST /v1/monitor/google call returns the page as typed JSON. These are the fields that matter for keyword research, from the OpenAPI reference:

A basic call

curl -X POST https://api.answerline.dev/v1/monitor/google \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "query": "invoice software for freelancers", "country": "US", "hl": "en", "include": { "aioverview": {} } }'

include.aioverview adds the AI Overview; {"markdown": true} inside it adds a markdown rendering. include.paaAioverview: true fills AI-Overview-type People Also Ask items with markdown and sources at no extra credit cost beyond the AI Overview add-on, though responses take longer.

Turning a response into a keyword record

The useful unit is one row per keyword per market per date, with the SERP reduced to features. This function does that in Python:

from urllib.parse import urlparse

def domain(url: str) -> str:
    host = urlparse(url).netloc.lower()
    return host[4:] if host.startswith("www.") else host

def keyword_record(keyword: str, country: str, result: dict) -> dict:
    organic = result.get("organicResults", [])
    aio = result.get("aioverview")
    paa = result.get("peopleAlsoAsk", [])
    top10 = [domain(r["link"]) for r in organic if r.get("position", 99) <= 10]
    return {
        "keyword": keyword,
        "country": country,
        "has_aio": aio is not None,
        "aio_source_domains": sorted({domain(s["url"]) for s in (aio or {}).get("sources", [])}),
        "paa_questions": [q["question"] for q in paa],
        "paa_answered_by_ai": sum(1 for q in paa if q.get("type") == "AIOVERVIEW"),
        "related": [r["query"] for r in result.get("relatedSearches", [])],
        "ads": len(result.get("ads", [])),
        "has_shopping": bool(result.get("shoppingCards")),
        "has_local": bool(result.get("localResults")),
        "has_knowledge_panel": bool(result.get("knowledgeGraph")),
        "has_discussions": bool(result.get("peopleAreSaying")),
        "top10_domains": top10,
        "distinct_top10_domains": len(set(top10)),
    }

Deriving intent from features

Features are Google’s own reading of intent, which beats guessing from the words. A simple, auditable classifier:

def intent(rec: dict) -> str:
    if rec["has_local"]:
        return "local"
    if rec["has_shopping"] or rec["ads"] >= 3:
        return "transactional"
    if rec["has_knowledge_panel"] and not rec["paa_questions"]:
        return "navigational"
    if rec["ads"] > 0:
        return "commercial"
    return "informational"

The thresholds are a starting point. Calibrate them on 50 keywords you have labelled by hand before trusting the output on thousands.

Expansion: from seeds to a keyword set

SERP data expands a seed list in three directions:

1. Related searches give modifiers and neighbouring topics (“invoice software for freelancers” → “free invoice template”, “best invoicing app uk”).
2. People Also Ask gives questions. Mining People Also Ask covers level-by-level expansion and clustering in depth.
3. AI query fan-out gives the searches AI assistants run before answering. Copilot’s response includes searchQueries, the web searches it ran while generating the answer; ChatGPT exposes its searches with include.searchQueries. Query fan-out explains why these are keyword candidates you will not find in volume tools.

A breadth-first expansion with a hard cap keeps cost predictable:

import os
import requests

API = "https://api.answerline.dev"
HEADERS = {"Authorization": f"Bearer {os.environ['ANSWERLINE_API_KEY']}"}

def serp(keyword: str, country: str) -> dict:
    resp = requests.post(
        f"{API}/v1/monitor/google",
        headers=HEADERS,
        json={"query": keyword, "country": country, "include": {"aioverview": {}}},
        timeout=360,
    )
    resp.raise_for_status()
    return resp.json()["result"]

def expand(seeds: list[str], country: str, limit: int = 50) -> dict[str, dict]:
    queue, records = list(seeds), {}
    while queue and len(records) < limit:
        kw = queue.pop(0).strip().lower()
        if kw in records:
            continue
        records[kw] = keyword_record(kw, country, serp(kw, country))
        queue.extend(r for r in records[kw]["related"] if r.lower() not in records)
    return records

This uses synchronous calls for readability. For more than a few dozen keywords, submit GOOGLE tasks with POST /v1/async/task/batch (up to 500 per request) and process results by webhook: async tasks cost 2 credits less each and do not fail when concurrency slots are busy. Rank tracking with async batches shows the submission and collection code.

Joining SERP data with volume

Once both sides exist, join on normalised keyword and market:

The combination answers questions neither side can alone:

• High volume, AI Overview present, few ads. Informational demand where an AI summary sits above the results; plan for being cited, not only for ranking. Monitoring AI Overviews covers tracking citations.
• Modest volume, shopping cards and many ads. Transactional and contested; paid and product feeds matter as much as content.
• Local pack present. Rankings vary by city; track with location targeting, see local rank tracking by city.
• Low distinct-domain count in the top 10. A few sites hold several positions; harder to break into than the volume suggests.
• Discussions present. Forum and social threads rank; community presence may be the faster route.

Keep volume and SERP snapshots dated. Volume is usually a monthly average; a SERP is a point-in-time observation that changes, so re-run SERP collection on a schedule rather than treating one snapshot as permanent. SERP feature change tracking turns repeated snapshots into events.

What it costs

Credit costs: a Google Search async task is 3 credits for one results page, each extra page adds 2, include.aioverview adds 2, and a synchronous call adds 2.

The free tier’s 500 credits a month cover 100 keywords with AI Overview detection as async tasks, enough to validate the classifier and the join. Plan prices are on /pricing.

Choosing a data source

1. You need to size demand: a volume source. This API cannot do it.
2. You need to know what ranking takes, what intent Google assigns, or whether AI Overviews answer the query: SERP data.
3. You need the questions and modifiers searchers see: People Also Ask and related searches from the SERP, plus AI query fan-out.
4. You need market-specific research: SERP data per country, hl and, for local queries, location. International keyword research covers multi-market runs.
5. You are building a keyword tool for customers: both, joined, with the SERP refreshed on a schedule.

Pitfalls

• Calling SERP features “volume”. Ads and PAA say something about intent, nothing about how many people search.
• Snapshot bias. One run is one observation; AI Overview presence in particular can come and go for the same query.
• Unbounded expansion. Related searches link to related searches forever. Cap by count and by depth, and dedupe on normalised text.
• Mixing markets. A keyword’s SERP in the US says little about the UK. Keep country in every key.
• Ignoring language. hl sets the interface language independently of the country; set it deliberately.
• Classifier drift. Re-check intent thresholds when Google changes layouts.

See the keyword research use case and the Google Search engine page to start.

Keep reading