2026-10-26, GEO · Fundamentals

Best LLM scraper APIs for 2027: engines, fields, prices

An LLM scraper API sends a prompt to a consumer AI product, such as ChatGPT, Gemini, Perplexity, Copilot, Grok or Google AI Mode, and returns the answer users see as JSON, with the pages it cites. This guide is for teams setting 2027 budgets for collection across several engines. It compares vendors on which engines each covers, whether one schema spans them, async delivery and webhooks, geo targeting, how each bills, and what a prompt set costs across all engines. Engine-specific fields are in the per-engine guides: ChatGPT, Perplexity, Gemini, Google AI Mode, Copilot and Grok. Dashboards that report on AI answers for you are in best AI visibility tools.

Facts about each vendor come from its own public pages, linked and checked on 2026-10-26; “not listed” and “not found” mean not on those pages on that date. These are October 2026 facts, not forecasts; re-check prices before you commit.

Engine coverage

No vendor page checked lists a scraper for the Claude, Meta AI or DeepSeek apps. SerpApi also has Brave AI Mode, Naver AI Overview and DuckDuckGo Search Assist APIs. ZenRows and Nimble were checked too: ZenRows documents no AI-answer product, and Nimble’s pricing lists search and agent APIs but no per-engine answer scraper.

Schema, delivery, geo and billing

Two terms used below:

• One schema: the same field name holds the same thing on every engine, so a parser for sources written for ChatGPT works on Perplexity.
• Billing unit: what one charge buys. A record (Bright Data), a results page (DataForSEO), a search (SearchApi, SerpApi), a credit (ScrapingBee), an event (Apify) and a request (AnswerLine) each map to one answer in the cost example below.

The vendors

Bright Data

Bright Data’s AI scrapers are separate scrapers in its Web Scraper API. Its ChatGPT page lists fields per engine, and they differ: Gemini and AI Mode return “Links attached” and “Citations”, Perplexity returns “Sources” and “Is shopping data”. The Grok page sells Grok at the same price, while the docs mark it “currently unavailable”, and the docs say Gemini is not available from European countries.

One price per record on every engine, success-only billing and 5K free records a month give it the lowest cost per answer in the cost example below. You write a field mapping per engine.

Fits data teams that want bulk datasets delivered by webhook or to cloud storage.

DataForSEO

The AI Optimization API has four products. LLM Scraper collects ChatGPT and Gemini “as they appear in supported interfaces”; LLM Responses “works through official LLM APIs”, so it returns model output, not the app’s answer; LLM Mentions is a database covering ChatGPT and Google AI Overview. Google AI Mode sits in the SERP API at the same per-page prices as LLM Scraper (pricing).

Results come as typed items with rank_group and rank_absolute, the same model as its Google results. The standard queue (up to 45 minutes) is the cheapest per answer here; live calls cost 3.3 times as much. Perplexity, Copilot and Grok are not covered as scrapers.

Fits SEO platforms already on DataForSEO that need ChatGPT, Gemini and AI Mode.

Oxylabs

Oxylabs runs each engine as a source in its Web Scraper API: chatgpt, gemini, perplexity, and Google AI Mode. The LLM sources docs say realtime and proxy endpoint integrations are not available for them, so results come by push-pull with a callback_url. The pricing page lists $1.50 per 1K results for ChatGPT, Perplexity and Gemini on every plan below Enterprise, and for AI Mode up to Basic.

Fits teams already on an Oxylabs plan who can work with callbacks.

SearchApi

SearchApi has a separate API for ChatGPT, Perplexity, Gemini, Bing Copilot, Google AI Mode and Google AI Overview, called with one key and the engine parameter. Billing is per successful search; the pricing runs from $4 per 1K on Developer to $1 per 1K on the 5M plan. No webhooks or async mode were found in its docs.

Six engines under one key and one monthly plan. Plans commit you to a monthly volume, and response shapes follow each engine.

Fits teams already using SearchApi for Google results.

SerpApi

SerpApi’s AI answer APIs cover search-engine answers: Google AI Mode, the Google AI Overview, Bing Copilot, Brave AI Mode, Naver AI Overview and DuckDuckGo Search Assist. AI Mode and Copilot both return text_blocks and references, the closest to one schema among the vendors here. It has no ChatGPT, Gemini or Perplexity API. The Copilot docs list no location parameter.

Fits teams tracking search-engine AI answers rather than chat assistants.

Apify

Apify Store is a marketplace of actors. Apify’s own actors cover ChatGPT (from $3.00 per 1,000), Gemini (from $3.00), Perplexity (from $9.00) and Google AI Mode (from $2.00), each with countryCode. Each actor has its own schema and price. On the free plan, access stops for the month once the $5 credit is used.

Fits prototypes, one-off studies and no-code workflows.

ScrapingBee

ScrapingBee has a ChatGPT API and a Gemini API, 15 credits per successful call each. Both docs say they cannot return citations 100% of the time. Its Grok scraper reads public shared-chat pages on grok.com, not answers to your prompt.

Fits teams already on ScrapingBee that need answer text from two engines.

Scrapeless

The Scrapeless docs list AI scrapers for ChatGPT, Perplexity, Microsoft Copilot, Gemini, Google AI Mode, Google AI Overview and Grok, and say only successful requests are billed. Its pricing page bills a prepaid balance and showed no AI scraper rates on 2026-10-26, so cost cannot be modeled from public pages.

Fits buyers who need seven engines from one vendor and will ask sales for rates.

Decodo

Decodo’s Web Scraping API targets include ChatGPT, Gemini and Perplexity templates and a Google AI Mode template, all on its premium proxy pool. Its product page lists plans from $19 a month but no price for the AI templates.

Fits teams already on Decodo’s scraping plans.

AnswerLine

AnswerLine has one endpoint per engine: /v1/monitor/chatgpt, /gemini, /perplexity, /copilot, /grok, /aimode, and /google with include.aioverview for the AI Overview. The same names hold the same things on every engine:

• text and, when requested, markdown.
• sources[] with position in display order, and citationPills[] on ChatGPT, Gemini, Perplexity, Copilot and AI Mode.
• provenance on every answer: capturedAt, requestedCountry, attempt, webSearchTriggered and model on chat engines.

Engine-specific fields keep their own names: inlineProducts[] on ChatGPT, shoppingCards[] on Gemini and AI Mode, related_queries[] on Perplexity, searchQueries[] and relatedQueries[] on Grok. Each engine page lists the fields its answers leave empty today.

Geo: ChatGPT, Gemini and AI Mode answer from 213 countries, Gemini also by US state, and AI Mode and Google Search down to the city by location; Perplexity, Copilot and Grok take any country, with no state.

Price per request at the Starter tier: ChatGPT $0.0006 and Gemini $0.0006, lower on larger tiers; Perplexity $0.0015, Copilot $0.00274, Grok $0.00218 and AI Mode $0.00206, one price on every tier; Google Search $0.00103, plus $0.00071 for the AI Overview, charged only when the page has one. A synchronous call costs the same as an async task, and failed requests are not charged (pricing docs). Across the five engines in the cost example, it costs more per answer than Bright Data’s records or DataForSEO’s standard queue.

Fits teams building AI-visibility or GEO products that need several assistants in one parser.

How to choose

1. List the engines your buyers use, then check the matrix. Bright Data, Scrapeless and AnswerLine list all of ChatGPT, Gemini, Perplexity, Copilot, Grok and AI Mode; Bright Data’s docs mark Grok unavailable, and Scrapeless publishes no AI scraper prices.
2. Count the parsers you will maintain. One product per engine means one mapping per engine, and a second vendor for the engines the first lacks adds another contract.
3. Check that each engine is live. Bright Data’s Grok and ScrapingBee’s Grok show how a listed engine can differ from what you need.
4. Pick a delivery model. Scheduled monitoring fits async jobs with webhooks or callbacks; agents and interactive tools need synchronous calls.
5. Check geo per engine. Country is common, but Bright Data excludes European countries on Gemini and SerpApi’s Copilot takes no location.
6. Convert billing units to cost per answer at your volume, with success-only billing and minimum top-ups included.

Cost example

100 prompts, US, collected daily for 30 days on ChatGPT, Gemini, Perplexity, Copilot and Google AI Mode: 3,000 answers per engine, 15,000 if all five are covered. List prices checked 2026-10-26, before taxes, discounts and one-time trials. Each line prices only the engines that vendor covers; recurring free allowances (Bright Data’s 5K records) are netted:

A vendor that covers three engines leaves you paying, integrating and mapping a second one for the rest. Add that work to the lower lines before comparing. AI monitoring cost planning covers budgeting across engines and cadences.

Daily collection across engines on AnswerLine

One async batch per day covers every engine, with a webhook for results:

import os, datetime, requests

API = "https://api.answerline.dev"
HEADERS = {"Authorization": f"Bearer {os.environ['API_KEY']}"}
today = datetime.date.today().isoformat()

prompts = ["best CRM for agencies", "HubSpot vs Pipedrive for small teams"]
engines = ["CHATGPT", "GEMINI", "PERPLEXITY", "COPILOT", "GROK", "AIMODE"]

tasks = [
    {
        "taskType": engine,
        "payload": {"prompt": p, "country": "US"},
        "idempotencyKey": f"crm-{i}-{engine}-US-{today}",
        "webhook": {"url": "https://example.com/hooks/answers"},
    }
    for i, p in enumerate(prompts)
    for engine in engines
]
tasks += [
    {
        "taskType": "GOOGLE",
        "payload": {"query": p, "country": "US", "include": {"aioverview": {"markdown": True}}},
        "idempotencyKey": f"crm-{i}-GOOGLE-US-{today}",
        "webhook": {"url": "https://example.com/hooks/answers"},
    }
    for i, p in enumerate(prompts)
]

r = requests.post(f"{API}/v1/async/task/batch", json=tasks, headers=HEADERS, timeout=30)
r.raise_for_status()
for item in r.json()["results"]:
    if not item["success"]:
        print(item["index"], item["error"]["code"])

A batch takes up to 500 tasks; split larger sets. Each finished task is POSTed to the webhook, with the result at response.result. Keys derived from prompt, engine, market and day make a retried submission safe. Verify the webhook signature before trusting the body; the batch pipeline guide covers retries and storage.

Pitfalls in multi-engine data

• Comparing engines with different fields. If one vendor returns ordered sources for ChatGPT and only answer text for Gemini, a citation-share gap between them reflects the vendor’s parser, not the engines.
• Mixing vendors per engine. Two vendors’ sources can differ in order, deduplication and what counts as a citation; store the vendor with each result.
• Assuming every market works on every engine. Check geo per engine; a vendor’s country list for one engine may not apply to another.
• Expecting an AI Overview on every query. Google shows none on many queries; AnswerLine charges the AI Overview only when the page has one.
• One run per prompt. Answers vary run to run on every engine; see AI answer volatility.
• Counting listed engines as live. Run your prompt set on each engine in a trial before you sign.

Every engine’s fields and countries are on the engines page, and the quickstart covers a first request.

Keep reading