Reliability
Reliability = 100 − mean(max − min) across 6 benchmarks. Higher is tighter. Inverted spread, not same-task repeatability.
26 models · Updated Oct 2026
Methodology View
Models
1GPT-6 Astra64.1
2Claude Opus 5.556.2
3GPT-6.1 Sol52.9
4Claude Sonnet 5.550.0
5GPT-5.6 Sol47.6
6Grok 4.647.4
7GPT-6 Sol46.3
8GPT-5.6 Terra45.6
9GPT-6.1 Terra45.1
10GPT-6 Luna40.6
11DeepSeek V4.1 Flash38.5
12Gemini 4 Argon38.0
13GLM-5.3 Flash37.5
14GLM-5.337.1
15Kimi K336.3
16GPT-5.6 Luna33.4
17GLM-5.231.7
18DeepSeek V4 Pro31.1
19Gemini 3.6 Flash29.8
20Claude Haiku 5.527.9
21Claude Opus 525.1
22DeepSeek V4 Flash23.7
23Gemini 3.1 Pro19.2
24Claude Opus 4.817.3
25Claude Sonnet 58.1
26MiniMax M37.7
Per-benchmark reliability (100 − spread) — higher is tighter
Box = 100 − (max − min) across 6 benchmarks. Higher is tighter. Inverted spread, not same-task repeatability.
| Rank | Models (26) | Status | |||||
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 64.1 | 66.8 | 55.9 | 80.2 | 21.2 | |
| 2 | Claude Opus 5.5 | 56.2 | 63.0 | 53.5 | 73.3 | 0.0 | |
| 3 | GPT-6.1 Sol | 52.9 | 66.5 | 30.9 | 73.5 | 0.0 | |
| 4 | Claude Sonnet 5.5 | 50.0 | 55.0 | 39.8 | 65.5 | 0.0 | |
| 5 | GPT-5.6 Sol | 47.6 | 38.3 | 29.0 | 58.1 | 23.9 | |
| 6 | Grok 4.6 | 47.4 | 44.6 | 37.8 | 61.0 | 0.0 | |
| 7 | GPT-6 Sol | 46.3 | 47.7 | 38.6 | 61.8 | 0.0 | |
| 8 | GPT-5.6 Terra | 45.6 | 42.3 | 32.1 | 49.6 | 14.8 | |
| 9 | GPT-6.1 Terra | 45.1 | 50.3 | 35.3 | 60.9 | 0.0 | |
| 10 | GPT-6 Luna | 40.6 | 39.7 | 20.9 | 62.4 | 0.0 | |
| 11 | DeepSeek V4.1 Flash | 38.5 | 38.0 | 36.6 | 51.2 | 0.0 | |
| 12 | Gemini 4 Argon | 38.0 | 31.7 | 0.0 | 77.2 | 0.0 | |
| 13 | GLM-5.3 Flash | 37.5 | 37.2 | 34.8 | 42.1 | 0.0 | |
| 14 | GLM-5.3 | 37.1 | 34.5 | 3.1 | 64.4 | 0.0 | |
| 15 | Kimi K3 | 36.3 | 32.6 | 15.5 | 55.0 | 0.0 | |
| 16 | GPT-5.6 Luna | 33.4 | 39.6 | 31.2 | 42.2 | 0.0 | |
| 17 | GLM-5.2 | 31.7 | 36.0 | 30.5 | 42.6 | 0.0 | |
| 18 | DeepSeek V4 Pro | 31.1 | 29.6 | 26.0 | 32.9 | 0.0 | |
| 19 | Gemini 3.6 Flash | 29.8 | 23.1 | 19.6 | 44.1 | 11.1 | |
| 20 | Claude Haiku 5.5 | 27.9 | 17.6 | 1.5 | 49.4 | 0.0 | |
| 21 | Claude Opus 5 | 25.1 | 20.9 | 5.1 | 42.8 | 0.0 | Cyber refused |
| 22 | DeepSeek V4 Flash | 23.7 | 21.6 | 17.3 | 34.5 | 0.0 | |
| 23 | Gemini 3.1 Pro | 19.2 | 21.4 | 5.0 | 24.5 | 0.0 | |
| 24 | Claude Opus 4.8 | 17.3 | 16.8 | 2.5 | 28.8 | 0.0 | |
| 25 | Claude Sonnet 5 | 8.1 | 4.6 | 0.7 | 14.1 | 0.0 | Cyber refused |
| 26 | MiniMax M3 | 7.7 | 3.3 | 0.0 | 11.0 | 0.0 |
Cyber refused: the model refused a material share of tasks on cybersecurity grounds. Refused tasks stay in its scores.
AI for the blue team.
Run Cotool's harness in your environment to get real security work done