Reliability
Reliability = 100 − mean(max − min) across 6 benchmarks. Higher is tighter. Inverted spread, not same-task repeatability.
16 models · Updated Sep 2026
Methodology View
Models
1GPT-6 Astra64.1
2GPT-5.6 Sol47.6
3Grok 4.647.4
4GPT-5.6 Terra45.6
5GLM-5.337.1
6Kimi K336.3
7GPT-5.6 Luna33.4
8GLM-5.231.7
9DeepSeek V4 Pro31.1
10Gemini 3.6 Flash29.8
11Claude Opus 525.1
12DeepSeek V4 Flash23.7
13Gemini 3.1 Pro19.2
14Claude Opus 4.817.3
15Claude Sonnet 58.1
16MiniMax M37.7
Per-benchmark reliability (100 − spread) — higher is tighter
Box = 100 − (max − min) across 6 benchmarks. Higher is tighter. Inverted spread, not same-task repeatability.
| Rank | Models (16) | |||||
|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 64.1 | 66.8 | 55.9 | 80.2 | 21.2 |
| 2 | GPT-5.6 Sol | 47.6 | 38.3 | 29.0 | 58.1 | 23.9 |
| 3 | Grok 4.6 | 47.4 | 44.6 | 37.8 | 61.0 | 0.0 |
| 4 | GPT-5.6 Terra | 45.6 | 42.3 | 32.1 | 49.6 | 14.8 |
| 5 | GLM-5.3 | 37.1 | 34.5 | 3.1 | 64.4 | 0.0 |
| 6 | Kimi K3 | 36.3 | 32.6 | 15.5 | 55.0 | 0.0 |
| 7 | GPT-5.6 Luna | 33.4 | 39.6 | 31.2 | 42.2 | 0.0 |
| 8 | GLM-5.2 | 31.7 | 36.0 | 30.5 | 42.6 | 0.0 |
| 9 | DeepSeek V4 Pro | 31.1 | 29.6 | 26.0 | 32.9 | 0.0 |
| 10 | Gemini 3.6 Flash | 29.8 | 23.1 | 19.6 | 44.1 | 11.1 |
| 11 | Claude Opus 5 | 25.1 | 20.9 | 5.1 | 42.8 | 0.0 |
| 12 | DeepSeek V4 Flash | 23.7 | 21.6 | 17.3 | 34.5 | 0.0 |
| 13 | Gemini 3.1 Pro | 19.2 | 21.4 | 5.0 | 24.5 | 0.0 |
| 14 | Claude Opus 4.8 | 17.3 | 16.8 | 2.5 | 28.8 | 0.0 |
| 15 | Claude Sonnet 5 | 8.1 | 4.6 | 0.7 | 14.1 | 0.0 |
| 16 | MiniMax M3 | 7.7 | 3.3 | 0.0 | 11.0 | 0.0 |
AI for the blue team.
Run Cotool's harness in your environment to get real security work done