Blog
Beyond CTFs: Evaluating AI Agents on Real Intrusion Data
The Security Infrastructure Powering EliseAI's Rapid Scale
Agents as Software
What does security look like in a world where intelligence is everywhere?
Engineering Blogs
Research Blogs
Beyond CTFs: Evaluating AI Agents on Real Intrusion Data
We benchmarked frontier models on a real macOS infostealer intrusion. This is not a CTF, which tend to test narrow, artificial scenarios. Tasks spanned incident response, threat hunting, and detection engineering.
Evaluating AI Agents in Security Operations (December 2025)
Which frontier model should you use for SecOps automation? We added the latest cohort of frontier models to our benchmark to find out.
Evaluating AI Agents in Security Operations
We benchmarked frontier AI models on realistic security operations (SecOps) tasks using Cotool’s agent harness and the Splunk BOTSv3 dataset. GPT-5 achieved the highest accuracy (63%), while Claude Haiku-4.5 completed tasks the fastest with strong accuracy. GPT-5 variants dominated the performance-cost frontier. These results provide practical guidance for model selection in enterprise SecOps automation.
Company Blogs
Agents as Software
What does security look like in a world where intelligence is everywhere?
April 15, 2026
.png?dpl=dpl_EguvJTzXB6cQHkAvykmvyTuUqPsY)
Announcing our $7.4m seed fundraising to build the AI Operating system for security teams
Cotool announces $7.4m seed round led by Andreessen Horowitz to build the AI Operating System for security teams
March 5, 2026
-1.png?dpl=dpl_EguvJTzXB6cQHkAvykmvyTuUqPsY)
Security without the Spectacle
Why would anyone want to start an AI security company?
May 12, 2025
Attackers are scaling with tokens
Cotool helps defenders operate at machine speed. See how security teams are scaling Detection & Response.
Book a demo
All Reads
Design and implement software systems, conduct code reviews, optimize application performance.
Agents as Software
What does security look like in a world where intelligence is everywhere?
Beyond CTFs: Evaluating AI Agents on Real Intrusion Data
We benchmarked frontier models on a real macOS infostealer intrusion. This is not a CTF, which tend to test narrow, artificial scenarios. Tasks spanned incident response, threat hunting, and detection engineering.
The Security Infrastructure Powering EliseAI's Rapid Scale
How EliseAI uses Cotool to automate investigations, expand detection coverage, and defend customer data across housing and healthcare
Announcing our $7.4m seed fundraising to build the AI Operating system for security teams
Cotool announces $7.4m seed round led by Andreessen Horowitz to build the AI Operating System for security teams
Evaluating AI Agents in Security Operations (December 2025)
Which frontier model should you use for SecOps automation? We added the latest cohort of frontier models to our benchmark to find out.
Evaluating AI Agents in Security Operations
We benchmarked frontier AI models on realistic security operations (SecOps) tasks using Cotool’s agent harness and the Splunk BOTSv3 dataset. GPT-5 achieved the highest accuracy (63%), while Claude Haiku-4.5 completed tasks the fastest with strong accuracy. GPT-5 variants dominated the performance-cost frontier. These results provide practical guidance for model selection in enterprise SecOps automation.
Context Management for Agentic Security
How we are solving the LLM Security Data problem
Security without the Spectacle
Why would anyone want to start an AI security company?