AI search data is a five-layer stack of 31 actors, and the leaders span several layers at once
Asked:
“who are the actors in the search data for ai space?”
A live web scan on 16 November 2026 mapped 31 distinct companies and organizations across 46 actor-layer pairs in five functional layers — AI-native search and research (10 actors), crawl and extraction (12), open web corpora (10), SERP data (7) and search-index APIs (7). Vertical integration is the pattern: Perplexity, Tavily, Firecrawl, Parallel, Jina AI, You.com and Bright Data each operate in more than one layer, and Common Crawl underpins the open-data substrate. It is a market map built from 23 independent source hosts, not an exhaustive census.
The ecosystem matrix: who plays where
Rows are actors, columns are the five market layers. Mark size hints at evidence coverage in this scan (pages found), not market share. Hover a row for products and pricing; multi-layer actors sit on top.
Layer sizes and their most-documented actor
Actor count per layer; layer counts sum to 46 because multi-layer vendors are counted in each layer they serve.
Seven vendors are vertically integrated across layers — Jina AI and Perplexity each appear in four of the five, Tavily and Parallel Web Systems in three.
docs.perplexity.ai
Common Crawl is the foundation of the open-corpus layer: it is the most-documented actor there (74 evidence pages), and derived corpora such as FineWeb, Dolma, RefinedWeb, C4, Pile-CC and RedPajama build on its crawls.
commoncrawl.org
Only two actors expose a general search-index API with published terms at the core of that layer — Brave and Perplexity — making the index itself the scarcest asset in the stack.
api-dashboard.search.brave.com
SERP pricing is a commodity race: Serper and ScrapingDog sell Google results from $0.29–$1.00 per 1,000 searches, while AI-native answer APIs charge up to $1,200 per 1,000 research calls.
firecrawl.dev
Method: live web research, 2026-11-16. 46 actor-layer rows covering 31 distinct actors in five functional layers, built from official documentation plus independent comparisons across 23 source hosts; actors kept only with evidence from at least two pages, and the most-used single URL backs 4 rows (8.7%). “Pages” counts evidence pages found in this scan — a coverage signal, not company size, revenue or product quality. A broad current-market map, not an exhaustive census; some vendors appear in multiple layers, so layer counts sum past 31. Long capability text is truncated in the table; hover the matrix for details.