Skip to main content
Brass calibration weights of different sizes lined up in a row, drawn in dots

Which AI models are best for security work?

How 26 AI models scored on BlueBench defensive security tasks: incident response, threat hunting, detection engineering, cost per task, and refusals.

Research brief
Updated 9 min read

Summary

On BlueBench, a public benchmark of 74 defensive security tasks, Claude Opus 5.5 had the highest overall score at 83.2, ahead of GPT-6 Astra at 80.3 and Gemini 4 Argon at 79.9. Every top model has a weak track or a cost catch: Claude Opus 5.5 trailed GPT-6 Astra on incident response and detection engineering, GPT-6 Astra scored 62.8 on threat hunting at $9.60 per task, the highest cost of the 26 models, and two earlier Claude models refused a material share of tasks.

Highest overall BlueBench score, Claude Opus 5.5
83.2
Cost per task for GLM-5.3 Flash, which scored 70.6
$0.11
Models tested on 74 defensive security tasks
26
Models flagged for refusing tasks on cybersecurity grounds
2

Key findings

  1. Claude Opus 5.5 had the highest overall BlueBench score at 83.2, followed by GPT-6 Astra (80.3), Gemini 4 Argon (79.9), and Grok 4.6 (79.2).
  2. No model led every track. GPT-6 Astra led incident response (88.0) and detection engineering (87.3); Claude Opus 5.5 led threat hunting (76.1) and malware analysis (89.4).
  3. Four models sit on the cost frontier, including Claude Sonnet 5.5 (77.5 at $1.01 per task) and GLM-5.3 Flash (70.6 at $0.11). Low cost can hide a weak track: DeepSeek V4.1 Flash scored 60.9 on incident response.
  4. Claude Opus 5 and Claude Sonnet 5 refused a material share of tasks on cybersecurity grounds and ranked 24th and 26th of 26.
  5. Gemini 4 Argon, third on capability, ranked 12th on reliability at 38.0, and Claude Opus 4.8 ranked 24th at 17.3. GPT-6 Astra had the tightest score spread (reliability 64.1).

What was measured

BlueBench is a benchmark for defensive security work, published by Cotool. Its October 2026 update tested 26 models on 74 tasks in six benchmarks, scored in four tracks: incident response, threat hunting, detection engineering, and malware analysis.

The six benchmarks come in two series:

  • Intrusion. Three datasets built on real hands-on-keyboard intrusions captured in production-grade telemetry, including EDR, host and cloud audit logs, and Zeek network metadata: a macOS investigation (14 tasks), a Windows enterprise intrusion (40 tasks), and an AWS cloud intrusion (8 tasks). The data comes from Threat Hunting Labs.
  • Simulation. Three generated enterprise environments with routine activity, a simulated intrusion, and benign lookalikes (5, 4, and 3 tasks). Because the environments are authored, none of the data can have leaked into training sets.

Each track is graded differently (Methodology). Incident response and threat hunting reports are graded by a single fixed LLM judge against a hidden rubric. Detection rules are re-executed against the evidence and scored on precision and recall, and a rule that fails to run scores zero. Malware analysis answers come from the Windows benchmark alone. The overall score weights incident response, threat hunting, and detection engineering at 30% each and malware analysis at 10%.

Every model runs in the same harness with the same system prompt, tools, and number of trials. Thinking mode is on where a model offers it, and refused or failed attempts stay in the averages. Cost is USD per evaluated task at list token prices with prompt caching, excluding the judge. Reliability is 100 minus the average, over the six benchmarks, of the gap between a model's highest and lowest scores within each one, which BlueBench describes as "inverted spread, not same-task repeatability."

Results

The top of the capability leaderboard, with two lower-cost models for comparison:

RankModelOverallIncident responseThreat huntingDetection engineeringMalware analysisCost per task
1Claude Opus 5.583.287.876.183.889.4$3.54
2GPT-6 Astra80.388.062.887.388.8$9.60
3Gemini 4 Argon79.986.874.580.772.5$6.76
4Grok 4.679.280.872.682.683.8$3.64
5Claude Sonnet 5.577.583.966.978.187.8$1.01
6GPT-5.6 Sol77.282.662.782.887.5$2.89
9DeepSeek V4.1 Flash71.560.968.581.781.9$0.17
12GLM-5.3 Flash70.672.663.274.774.7$0.11

Every top model has a weak track. No model led every track, and the gaps inside a single model are larger than the gaps between the top four overall:

  • Claude Opus 5.5 trailed GPT-6 Astra on incident response (87.8 against 88.0) and detection engineering (83.8 against 87.3), and Claude Opus 4.8 outscored it on detection engineering at 85.3.
  • GPT-6 Astra scored 62.8 on threat hunting, tenth on that track and more than 13 points below Claude Opus 5.5, and its $9.60 per task was the highest cost of the 26 models.
  • Gemini 4 Argon scored 72.5 on malware analysis, against 89.4 for Claude Opus 5.5, and ranked 12th on reliability.
  • Grok 4.6 scored 80.8 on incident response, seventh on that track.
  • Claude Sonnet 5.5 scored 66.9 on threat hunting, 9.2 points below Opus 5.5.

Cost varies more than score. Four models sit on the cost frontier, where no other model scores higher for less: Claude Opus 5.5, Claude Sonnet 5.5, DeepSeek V4.1 Flash, and GLM-5.3 Flash. Claude Sonnet 5.5 scored 5.7 points below Opus 5.5 at under a third of the cost per task. DeepSeek V4.1 Flash and GLM-5.3 Flash, both among the eight open-weight models BlueBench evaluated (Simulation-002, Simulation-003), scored 71.5 and 70.6 at $0.17 and $0.11 per task. Their weak spots are in report-writing tracks: DeepSeek V4.1 Flash scored 60.9 on incident response and GLM-5.3 Flash 63.2 on threat hunting.

Reliability doesn't track capability. On the reliability leaderboard, GPT-6 Astra had the tightest spread (64.1), followed by Claude Opus 5.5 (56.2). Gemini 4 Argon ranked 12th at 38.0 and Claude Opus 4.8 ranked 24th at 17.3. The spread includes differences in task difficulty, not only repeated trials, so read it as a measure of consistency across tasks rather than run-to-run variation. Run-to-run consistency on your own alerts has to be measured in production, for example by grading every run, as Cotool's AI judge does (see How Cotool uses these results).

Two models were flagged for refusals. Claude Opus 5 and Claude Sonnet 5 refused a material share of tasks on cybersecurity grounds and ranked 24th (50.5) and 26th (44.8). Claude Sonnet 5 still scored 86.9 on malware analysis but 29.2 on incident response and 23.8 on threat hunting. The newer Claude Opus 5.5 and Claude Sonnet 5.5 ranked first and fifth.

What BlueBench doesn't cover

  • Offensive work. BlueBench measures defense only. Offensive benchmarks answer a different question: Cybench, for example, scores agents on 40 professional-level capture-the-flag tasks that involve finding vulnerabilities and executing exploits.
  • Other harnesses and prompts. Every model ran with the same harness and prompt, with nothing tuned per model. A vendor's agent, tools, or prompting can move a model's results in either direction.
  • Exact ordering. Report scores depend on an LLM judge, and the benchmark pages note that "close scores do not establish a statistically significant ordering." A gap of a point or two, such as Gemini 4 Argon's 79.9 against Grok 4.6's 79.2, shouldn't be read as a real ranking.
  • Your environment. The simulations have three to five tasks each, and none of the benchmarks use your log sources or alert mix.

What it means for security teams

Pick models per job rather than one model for everything. A model's detection engineering score says little about its investigation reports: the table above shows gaps of more than 20 points between tracks for some models, such as DeepSeek V4.1 Flash's 81.7 on detection engineering against 60.9 on incident response. For high-volume work such as first-pass triage, the two open-weight models on the cost frontier scored 11.7 and 12.6 points below the top model overall, and most of that gap sits in incident response and threat hunting, the tracks that grade investigating an alert and reporting what happened. Check the track that matches the job, not the overall rank.

In practice that means a model setting per agent rather than per platform. In Cotool, for example, the model is set on each agent, so a team could run its triage agent on GLM-5.3 Flash and its hunt agent on Claude Opus 5.5, which led threat hunting at 76.1. At BlueBench's per-task costs, 1,000 triage tasks would cost about $110 on GLM-5.3 Flash, $3,540 on Claude Opus 5.5, and $9,600 on GPT-6 Astra. A benchmark task is a full investigation rather than a single alert, so read those figures as a ratio, not a budget.

Test for refusals on your own work before you commit to a model. The two Claude models flagged for refusals ranked near the bottom of the table, while newer models from the same family ranked first and fifth, so a refusal problem can appear or disappear with a model update. Cotool reports the same pattern outside BlueBench: when it evaluated Claude Fable 5 on the 40 tasks of the Windows intrusion dataset, Anthropic's guardrails blocked every request. Because a refusal can arrive with any update, plan the fallback before you need it, whether that is a second provider, an open-weight model, or an abliterated model with its refusal behavior edited out. Cotool Router is one worked example: when a call is refused or a provider fails, it classifies the error, discards the incomplete output, and retries the same turn on another provider or an open-weight model, never on a model subject to the same provider-specific failure, and it bills the usage to the model that finished the work.

Then test on your own data. BlueBench gives you a shortlist, not a decision; how to evaluate an AI SOC platform covers running a bake-off on your own alerts. For how the two leading chat apps compare for security teams, see Claude vs ChatGPT for cybersecurity, and for how these models are used in practice, see what an AI SOC is.

How Cotool uses these results

Cotool publishes BlueBench and uses it in its own product. Its AI SOC lets you set the model for each agent, from Anthropic, OpenAI, Google, xAI, open-weight providers, or your own endpoint, and recommends a starting model for each agent's job from these results.

The per-track gaps above are the reason the setting is per agent. Every Cotool run is graded by an AI judge against your acceptance criteria, so after a model switch the eval scores on your own alerts show whether it helped, which is the test BlueBench can't run for you. The same judge flags a broken integration, such as an expired Okta token, as soon as a run fails on it.

FAQ

Which AI model is best for cybersecurity?
It depends on the task. On BlueBench's defensive security tasks, Claude Opus 5.5 had the highest overall score (83.2), GPT-6 Astra led incident response and detection engineering, and Claude Sonnet 5.5 scored 77.5 at $1.01 per task. BlueBench does not test offensive work such as exploitation, so for penetration testing, look at offensive benchmarks such as Cybench instead.
Which Claude model is best for cybersecurity?
On BlueBench, Claude Opus 5.5 scored highest of all 26 models tested (83.2 at $3.54 per task). Claude Sonnet 5.5 scored 77.5 at $1.01 and Claude Haiku 5.5 scored 68.5 at $0.59. Claude Opus 5 and Claude Sonnet 5 refused a material share of tasks on cybersecurity grounds and ranked near the bottom.
Is Claude better than ChatGPT for cybersecurity?
Comparing the underlying models on BlueBench, Claude Opus 5.5 (83.2) scored above GPT-6 Astra (80.3), the highest-scoring OpenAI model, overall. Claude Opus 5.5 trailed GPT-6 Astra on incident response and detection engineering and on reliability. GPT-6 Astra scored 62.8 on threat hunting, against 76.1, and cost $9.60 per task against $3.54.
What is the best open-weight model for cybersecurity?
BlueBench evaluated eight open-weight models, and the top one varies by benchmark: GLM-5.3 led them on the Windows (78.2%) and AWS (90.2%) intrusions, DeepSeek V4 Pro on the macOS investigation (90.2%), and DeepSeek V4.1 Flash on two of the three simulations. Overall, DeepSeek V4.1 Flash scored 71.5 at $0.17 per task and GLM-5.3 Flash scored 70.6 at $0.11.

Sources

  1. BlueBench — AI Benchmark for Defensive Security · accessed November 16, 2026
  2. BlueBench Capability Leaderboard · accessed November 16, 2026
  3. BlueBench Reliability Leaderboard · accessed November 16, 2026
  4. BlueBench Cost Efficiency · accessed November 16, 2026
  5. BlueBench Methodology · accessed November 16, 2026
  6. Windows Enterprise Intrusion · accessed November 16, 2026
  7. AWS Cloud Intrusion · accessed November 16, 2026
  8. macOS Threat Investigation · accessed November 16, 2026
  9. Initial Access & Command-and-Control · accessed November 16, 2026
  10. Identity & Active Directory Attacks · accessed November 16, 2026
  11. Impact & Exfiltration · accessed November 16, 2026
  12. Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models · accessed November 16, 2026
  13. AI SOC Without the Black Box · accessed November 16, 2026
  14. Introducing Cotool Router · accessed November 16, 2026

Keep reading

Attackers are scaling with tokens

Cotool helps defenders operate at machine speed. See how security teams are scaling Detection & Response.

Book a demo