Google's Gemini 4 Carbon has taken the recent top spot on a major independent AI intelligence index. It also leads a defensive cybersecurity benchmark that asks agents to find vulnerabilities and fix the code without breaking it.
Those are good reasons to pay attention. They aren't enough to declare your current AI model obsolete.
Claude still leads a prominent coding-agent comparison. GPT-6 Astra beats Carbon on one software-engineering benchmark in Google's own comparison table. And some of Carbon's most impressive numbers come from tests with conditions that matter a great deal outside a chart.
For anyone choosing models for engineering, research or business workflows, Carbon gives us a useful new contender and a surprisingly clear lesson in how to read the frontier.
Let’s unpack the announcement
Google announced Gemini 4 Carbon on October 28, following Gemini 4 Argon earlier this month. Carbon leads Artificial Analysis Intelligence Index v5 at 59, ahead of Claude Opus 5.5 at 58 and GPT-6 Astra at 57.
Its strongest evidence includes scientific reasoning, long-context graph reasoning and defensive security. Coding and business-work results are more mixed.
Access is the immediate limitation. Google's announcement starts with selected cyber defenders and a gradual expansion through trusted Fairwind partners. Broad developer, business and consumer availability is still coming. Treat this as an announced frontier model with restricted rollout, not a model everyone can deploy today just yet.
Scores below are published evaluations, not my own hands-on tests. Rankings can and will change. Benchmarks and real-world usage are two entirely different things.
The headline win: 59 against 58 and 57
Artificial Analysis's Carbon evaluation puts Carbon first on its updated Intelligence Index. The comparison uses Carbon at high reasoning and Opus 5.5 and Astra at maximum reasoning.
Carbon leads four of the index's nine evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience and AutomationBench-AA. That is a broad result, rather than a model taking first place on one narrow test.
The index itself changed on October 27. Intelligence Index v5 gives equal weight to knowledge work, coding, science and general capabilities. Sixty percent of its weight now comes from evaluations with private test sets or held-out answers.
A score of 59 is an index score, not 59% accuracy. Nor does a one-point lead establish that Carbon will reliably outperform Opus on your assignment. Comparisons with older index versions would be misleading.
The useful conclusion is that Google has a credible contender at the top of this particular broad evaluation. The individual results tell you where to investigate next. And it's important to test and refine this yourself with your own workflows.
Coding: Claude keeps an important lead
Artificial Analysis's Coding Agent Index scores Carbon in Antigravity CLI at 59, behind Opus 5.5 in Claude Code at 62.
That comparison evaluates the model working inside an agent application. Its tools, prompts and execution environment affect the result. It is closer to choosing a working coding assistant than a model-only exam, but it doesn't isolate the model's contribution.
Google's published evaluation table and methodology also show Carbon losing on FrontierSWE 2:
- GPT-6 Astra: 65.5%.
- Claude Opus 5.5: 62.3%.
- Gemini 4 Carbon: 61.2%.
- Gemini 4 Argon: 55.0%.
Carbon improves on Argon without taking the lead. Google's comparison draws on published results rather than rerunning every rival under one identical setup.
There may also be a reporting trap here. Terminal-Bench 4.0 appears with different Carbon scores across Google's table, Artificial Analysis's coding-agent article and Vals AI. Those rows should stay attached to their evaluator and configuration, averaging them or selecting the highest would hide what was measured.
For a development team, Carbon deserves a trial on difficult debugging and maintenance work when access becomes available. The evidence doesn't justify replacing a working Claude Code or Codex setup purely because Google won the overall index.
The security result I would examine first
The CWE-bench v1 leaderboard is especially interesting because it tests a complete defensive assignment. An agent receives a repository and must audit and fix it. The programmatic checks require the exploit to stop working while existing functionality still passes its tests.
Its published pass@1 scores are:
- Gemini 4 Carbon: 71%.
- Grok 4.7, Gemini 4 Argon and GPT-6 Astra: 68% each.
- Claude Opus 5.5: 67%.
The benchmark uses 120 held-out tasks. Each model gets four rollouts per task, with a one-hour limit for each rollout; pass@1 and pass@4 are reported separately. The agents use different harnesses, including Antigravity, Codex and Claude Code.
Carbon's lead is encouraging. So is the definition of success: a working repair, not a convincing explanation of what someone else should fix. But a 71% pass@1 score still leaves a substantial failure rate. It is not evidence that an agent can safely patch production without review.
Google also reports 88.2% on SEC-bench Pro versus Astra's 85.4%, and a headline 54.8% on ExploitGym versus Astra's 42.4% and Opus's 34.5%.
Those are Google-published comparisons with important qualifications. SEC-bench Pro uses a 183-task JavaScript-engine split. Google's cybersecurity launch results were evaluated without guardrails. Its ExploitGym methodology includes both six-hour and extended ten-hour task limits, so the headline should not be treated as a proven equal-budget comparison with every rival.
My takeaway is to test whether Carbon can reduce the work between a security finding and a reviewable, regression-tested fix. That is a much more useful business question than whether an agent can produce an impressive exploit demonstration.
Long context: a result worth watching closely
One of the largest gaps in Google's table is GraphWalks, a synthetic test of reasoning over graph information in long inputs.
On the 256,000-to-one-million-token subset, Google reports breadth-first-search F1 scores of:
- Gemini 4 Carbon: 96.2%.
- Gemini 4 Argon: 84.2%.
- GPT-6 Astra: 71.8%.
- Claude Opus 5.5: 66.8%.
Google computed these results for all the listed models. That makes the comparison useful, while still leaving it a vendor-run evaluation. So there is potentially bias in that ranking.
Artificial Analysis lists Carbon with a one-million-token context window and text, image, video and speech inputs, with text output. The GraphWalks result adds evidence about using long inputs, beyond simply accepting them.
It does not establish 96.2% accuracy on contracts, financial reports or large codebases. A graph-traversal benchmark and an inconsistent collection of business documents are different problems.
A sensible test would give Carbon a long document set containing a revised requirement and an obsolete version. Ask it to produce an answer with citations, explain which version controls and identify missing evidence. That would test something a business can recognize.
Business work has several winners
Vals AI's independent evaluation places Carbon second on Vals Index v2.2:
- Claude Sonnet 5.5: 70.85%.
- Gemini 4 Carbon: 70.36%.
- Claude Opus 5.5: 70.18%.
Those are close results. Vals displays Carbon's uncertainty as ±0.97, so I would not turn its small numerical advantage over Opus into a confident purchasing recommendation.
Carbon also ranks first on Legal Research Bench at 55.77%. Being first at that score is a reminder of how difficult the work remains, not a reason to remove legal review.
On Artificial Analysis's AA-Briefcase knowledge-work evaluation, Carbon improves to 1609 Elo from Argon's 1483. It still trails the article's reported results for Grok 4.7 at 1638, Muse Wonder at 1659 and Kimi K3.1 Pro at 1678. These are benchmark preference ratings, not accuracy percentages.
Even computer use splits the field. Google's table puts Carbon at 81.5% on the OSWorld 2.1 offline subset, versus Astra at 80.9% and Opus at 87.2%. This is an 82-task subset, not the full benchmark, and Google's Gemini scores take the maximum across three runs.
Research, presentations, software changes and desktop interaction should be separate evaluation categories. Buying one model for all of them because it tops a composite leaderboard skips the most useful evidence.
The hallucination number needs its definition
Artificial Analysis reports an AA-Omniscience v1.1 score of 54 for Carbon versus 48 for Opus 5.5. The benchmark rewards correct answers and penalizes incorrect ones while allowing the model to abstain.
Carbon's reported accuracy is 58%, and its benchmark hallucination rate is 11%. Argon has a lower hallucination rate at 4%, while Opus 5.5 and Astra are each reported at 50% in the Carbon analysis.
That does not mean half of everything Claude or GPT says is false. These are measurements on a deliberately challenging factual-knowledge benchmark, with abstention affecting the interpretation. They should not be transplanted into a claim about everyday conversations or source-grounded enterprise workflows.
The interesting tradeoff is within Google's own models:Carbon answers more questions correctly than Argon, but its hallucination rate rises. Improved capability does not automatically improve every reliability measure.
In a world where Trust is becoming extremely important in the world of AI, this is very important to consider.
Google separately reports 0.0% successful attacks in its Gray Swan indirect prompt-injection evaluation. That means no successful attacks in the evaluated set, not immunity to future attacks or permission to remove access controls.
Pricing: read past the introductory rate
Google says Carbon will launch at $2 per million input tokens and $10 per million output tokens. After the introductory period, the stated prices are $4 and $20. Cached input receives a 95% discount from the input price. The launch sources do not give a confirmed end date for the introductory period.
Artificial Analysis uses standard pricing in its cost comparisons. Its reported cost per Intelligence Index task is:
- Gemini 4 Carbon: $7.76.
- Claude Opus 5.5, maximum reasoning: $10.04.
- GPT-6 Astra, maximum reasoning: $6.25.
Carbon is cheaper than Opus on that measurement and more expensive than Astra. It is not automatically the economical choice just because its introductory token prices look attractive.
The cheaper frontier options also deserve attention. Artificial Analysis's October 27 index report lists GPT-6.1 Sol at 55 index points and $1.27 per task, and GPT-6.1 Terra at $0.96 per task. These are benchmark-specific costs, not promises of equivalent quality or production savings.
For a business decision, record the cost of an accepted result, including retries and human corrections. A lower API bill can disappear quickly if a reviewer has to repair the output.
And a good best-practice is to use more expensive models for reasoning and orchestration and use the more affordable models as the work-horses to optimize token spend.
Your AI Pathfinder Action Plan
Prepare a small comparison around work you already understand. If Carbon access isn't available, establish the baseline now and add it later.
- Select an engineering task, a source-grounded research task and a document-production task. Define what a reviewer would accept before running any model.
- Record the exact model, reasoning effort, agent application, tools and time limit. Compare complete products if that is what you intend to buy; use a common harness if you want to investigate the models themselves.
- Track accepted results, elapsed time, API spend and correction time. Log unsupported claims and failed tests instead of counting an agent's completion message as success.
- Keep tests in a sandbox with scoped access. Require human approval before merging code, publishing, sending messages or changing production.
Frequently Asked Questions
Can I use Carbon today?
Google announced restricted initial access for selected cyber defenders, with trusted-partner expansion through Fairwind. The launch does not establish broad public availability or access through your existing subscription.
Is Carbon the best model for coding?
It leads the broad AA Intelligence Index, but Opus 5.5 in Claude Code leads the cited Coding Agent Index. Astra also leads Carbon on FrontierSWE 2 in Google's table. Choose the test that resembles your work.
Does the security performance make it safe to run autonomously?
No. Defensive capability, prompt-injection resistance and permission to act are separate questions. Keep review, least-privilege access and rollback requirements even when benchmark performance improves.
The Bottom Line
Carbon's security and long-context results make it a serious evaluation candidate. Its independent overall ranking supports that interest, while its losses in coding, knowledge work and computer use help narrow where to start.
I would prioritize a difficult, reviewable engineering or research assignment where the current model struggles. Keep the incumbent as the baseline and require Carbon to earn the switch with accepted work, measured cost and fewer corrections.
For more on choosing between the major model families, read GPT-6 Astra, Claude or Gemini: Which AI Is Worth Paying For?
About Jason Fleagle
Jason Fleagle is Head of AI for Netsync and writes AI Pathfinder to help leaders make practical decisions about AI strategy, governance and implementation. You can learn more at thejasonfleagle.com and netsync.com.
