“Google Carbon 4” is read as Google DeepMind’s Gemini 4 Carbon. This revision keeps the earlier report’s evidence — 85 rows from 24 URLs across 6 hosts (14 official paired comparisons, 70 independent observations, 1 official specification; no URL backs more than 17.6% of rows) — and turns it into ten charts: Google first-party comparisons in blue vs gray, independent evaluations (LMArena, Artificial Analysis, Vals AI) in violet, and the official API economics last.
First-party claims from Google’s MKA technical paper and launch blogs. Dots: Carbon comparator. Benchmarks are faceted — upper facet is higher-is-better (win/score rates), lower facet is lower-is-better (attack success rates, where a smaller dot is safer). Hover a row for exact displays.
Source: storage.googleapis.com · blog.google — first-party Google results
Positive bars favor Carbon after flipping lower-is-better metrics. Units are points of each metric’s own scale — percentage points only where the metric is a percentage; never comparable across benchmarks.
Source: storage.googleapis.com · blog.google — first-party Google results
Google’s MKA paper reports ± intervals. Where bars overlap within error — Carbon vs Claude Opus 5.5 nearly everywhere — treat the gap as close, not decisive.
Source: storage.googleapis.com — first-party Google result
Independent, crowd-voted. Carbon (High) ranks 1 in Overall, Coding, Hard Prompts and Instruction Following; 3 in Creative Writing; 4 in WebDev. Whiskers show the leaderboard’s displayed ± uncertainty; positions and intervals move as votes arrive.
Source: lmarena.ai — independent evaluation, snapshot of 2026-10-27
Independent. Left: Intelligence Index by thinking level. Right: High vs Medium across benchmarks in each benchmark’s own unit — High wins coding and knowledge, Medium wins CritPt Hard physics and GDP.pdf, and AutomationBench ties. More thinking is not uniformly better.
Source: artificialanalysis.ai — independent evaluation
Independent measurements, each panel on its own unit. High takes 3.3× longer end to end and costs $7.76 per Intelligence Index task vs $6.59 for Medium, for two extra index points.
Speed & cost: artificialanalysis.ai — independent. Pricing & context: ai.google.dev — official. Output draws on the remaining unified 1,048,576-token window after input; input and output cannot both be maxed at once.
Choose Carbon for difficult, high-value, long-horizon agentic coding, cybersecurity, scientific reasoning and complex multimodal work — the places where its rank-1 arena results, MKA leads and safety margins pay for its latency and token appetite. Gemini 4 Argon remains the sensible default for routine or high-volume work unless customer-specific evaluations justify Carbon’s additional token use and effective cost — especially after the December price doubling.
| Type | Evaluator | Category | Metric | Config | Carbon | Comparator | Rank | Source |
|---|
Method: 85 rows gathered from 24 URLs across 6 hosts (blog.google, ai.google.dev, storage.googleapis.com, lmarena.ai, artificialanalysis.ai, vals.ai); no URL supplies more than 17.6% of rows. 14 official paired comparisons and 1 official specification are Google first-party; 70 observations are independent (LMArena snapshot 2026-10-27, Artificial Analysis, Vals AI). Scores are in each benchmark’s own unit (%, Elo-style arena score, index points, seconds, tokens/s, USD); heterogeneous scores are never averaged, ranks are ordinal, and High/Medium/Low thinking configurations are kept separate. Direction-adjusted advantage flips lower-is-better metrics. Cut for space: per-row published timestamps and page titles (links retained).