Gemini 4 Carbon leads most benchmarks it touches — but more thinking is not uniformly better, and December doubles the price

Asked:
“do a good research on google carbon 4 and it should be data driven inculding benchmarks and the graphs and comparisons also”
Asked (follow-up):
“use more graphs if you can”

“Google Carbon 4” is read as Google DeepMind’s Gemini 4 Carbon. This revision keeps the earlier report’s evidence — 85 rows from 24 URLs across 6 hosts (14 official paired comparisons, 70 independent observations, 1 official specification; no URL backs more than 17.6% of rows) — and turns it into ten charts: Google first-party comparisons in blue vs gray, independent evaluations (LMArena, Artificial Analysis, Vals AI) in violet, and the official API economics last.

Official paired comparisons — Carbon vs its rivals, by Google

First-party claims from Google’s MKA technical paper and launch blogs. Dots: Carbon comparator. Benchmarks are faceted — upper facet is higher-is-better (win/score rates), lower facet is lower-is-better (attack success rates, where a smaller dot is safer). Hover a row for exact displays.

Source: storage.googleapis.com · blog.google — first-party Google results

Direction-adjusted advantage

Positive bars favor Carbon after flipping lower-is-better metrics. Units are points of each metric’s own scale — percentage points only where the metric is a percentage; never comparable across benchmarks.

Source: storage.googleapis.com · blog.google — first-party Google results

MKA agentic suite — win/score rates with uncertainty

Google’s MKA paper reports ± intervals. Where bars overlap within error — Carbon vs Claude Opus 5.5 nearly everywhere — treat the gap as close, not decisive.

Source: storage.googleapis.com — first-party Google result

LMArena — rank 1 in four of six text boards

Independent, crowd-voted. Carbon (High) ranks 1 in Overall, Coding, Hard Prompts and Instruction Following; 3 in Creative Writing; 4 in WebDev. Whiskers show the leaderboard’s displayed ± uncertainty; positions and intervals move as votes arrive.

Source: lmarena.ai — independent evaluation, snapshot of 2026-10-27

Artificial Analysis — thinking level changes the answer

Independent. Left: Intelligence Index by thinking level. Right: High vs Medium across benchmarks in each benchmark’s own unit — High wins coding and knowledge, Medium wins CritPt Hard physics and GDP.pdf, and AutomationBench ties. More thinking is not uniformly better.

Source: artificialanalysis.ai — independent evaluation

Speed and cost — the price of High thinking

Independent measurements, each panel on its own unit. High takes 3.3× longer end to end and costs $7.76 per Intelligence Index task vs $6.59 for Medium, for two extra index points.

Speed & cost: artificialanalysis.ai — independent. Pricing & context: ai.google.dev — official. Output draws on the remaining unified 1,048,576-token window after input; input and output cannot both be maxed at once.

Google’s own safety pairing is the starkest gap: Carbon passes the 375-task Agentic Honesty Benchmark at 92% vs Argon’s 70%, and cuts Gray Swan attack success from 9.86% to 2.17%. blog.google
On MKA, Carbon beats GPT-6 Astra on all three win/score rates but loses Climb-WSR (54.34 vs 56.56) and Overall (71.89 vs 73.26) to Claude Fable 5.2 — a first-party admission of a rival lead. storage.googleapis.com
Vals AI independently scores Carbon 89.1% on Vibe Code Bench, 80.1% on Code Migration and 67.2% on Finance Agent v2.1, backing the agentic-coding story outside Google’s own numbers. vals.ai
Introductory API pricing runs only through November 30, 2026; every December rate is exactly double — input $2→$4, output/thinking $10→$20, cached input $0.10→$0.20 per million tokens. ai.google.dev

Recommendation

Choose Carbon for difficult, high-value, long-horizon agentic coding, cybersecurity, scientific reasoning and complex multimodal work — the places where its rank-1 arena results, MKA leads and safety margins pay for its latency and token appetite. Gemini 4 Argon remains the sensible default for routine or high-volume work unless customer-specific evaluations justify Carbon’s additional token use and effective cost — especially after the December price doubling.

All 85 evidence rows

TypeEvaluatorCategoryMetricConfigCarbonComparatorRankSource

Method: 85 rows gathered from 24 URLs across 6 hosts (blog.google, ai.google.dev, storage.googleapis.com, lmarena.ai, artificialanalysis.ai, vals.ai); no URL supplies more than 17.6% of rows. 14 official paired comparisons and 1 official specification are Google first-party; 70 observations are independent (LMArena snapshot 2026-10-27, Artificial Analysis, Vals AI). Scores are in each benchmark’s own unit (%, Elo-style arena score, index points, seconds, tokens/s, USD); heterogeneous scores are never averaged, ranks are ordinal, and High/Medium/Low thinking configurations are kept separate. Direction-adjusted advantage flips lower-is-better metrics. Cut for space: per-row published timestamps and page titles (links retained).

This report was generated automatically by Keenable SELECT at a user's request, from publicly available web sources linked herein. Keenable does not review, verify, or endorse its contents and makes no representation as to accuracy, completeness, or timeliness; AI-based extraction may contain errors. Nothing in this report is investment, legal, financial, or other professional advice. All trademarks and referenced content remain the property of their respective owners; no affiliation or endorsement is implied. To report an error, rights concern, or request removal: legal@keenable.ai.

Keenable SELECTAsk your own question
Made with Keenable SELECT